 Well, good evening everybody, so my name is Andrew and I've been a somewhat irregular visitor to these evenings. I would describe myself as a fairly casual ember developer over the last 18 months. And consider this talk as somewhere between a demonstration of an idea, but not so much necessarily for demonstrating the techniques as an opportunity for group therapy to see if this actually is a sensible thing to do by any measure. And if I've chosen an at all sensible way to do it. So the title here is first step towards polymorphic validation with ember and rails. Perhaps I just put polymorphic in because it makes everything sound better. And as a bit of background, my sort of computing background, I don't know how many this aligns with, started with AmigaBasic circa 1992, which I just discovered today was actually made by Microsoft when I looked up the YouTube video, so that was a bit of a surprise. And then graduating to VisualBasic 4 in about 1995, which gave me endless hours of fun. Now, my day job, I'm not a developer at all. My day job is a pediatric doctor. And that does give me the opportunity to interact with lots of forms of technology. Principally, telephones from the 1980s, which you can create a fairly handy hands free kit from when you're on the telephone to IT support for 30 minutes waiting for an answer because your computer's broken. And fax machines, which we still make an awful lot of use of. So there you go. In the words of Ron Burgundy, it is amateur hour. So to outline the talk, I'm going to introduce the general pattern for validation in rails and ember using existing tools. State the problem that I've encountered, my proposal to solve it. And then we can think of the limitations and future directions it could go in if it isn't a completely mad idea. So for validation, we all know that we need surface-eyed validation first and foremost because we can't trust anything that comes from the client. But the whole point of tools like ember is trying to allow us to have very responsive client-side validation that allows people to know whether they're entering the right things before they ever get to the submit button and that reduce network traffic possibly. So ideally, we'd like both. Now at the moment, if you're using a combination of rails and ember, which is very popular, then you may be validating your server-side models using active record validations for which we just run these simple validation methods within the class definition and can incorporate quite a range of different types of validation. I got into rails probably about eight years ago, but I don't think I'd ever really understood Ruby properly. And it was only when doing this I began to realise that I was actually simply running a command with that statement that it wasn't some sort of magical, magical declaration. So there you go. And then in ember, we can use ember validations, which unsurprisingly is based on active record validations and therefore looks very similar. Significantly in this case, actually, the validations is applied as an extension to a component or a controller, not to a model directly. And so we simply include the ember validations component, and then we can give a hash of validation strategies, which will be applied. And as far as I can tell, the way that they're defined is almost identical to the way that they are in rails validations. So the problem statement is that it requires us to declare our validations on the server side and the client side. Now that's not such a big deal when you only have one set of validation rules per model type, because we already do that for models. We declare our models on the client, we declare our models on the server. We don't get too upset about that, although it does cause problems if you want to change your model specifications and you forget to do one or the other side. So it just multiplies our work by two. Every time you want to define or add a validation, we simply define it on both ends. So maybe it's not such a big problem. But what about models where we have some sort of conditional or type specific validation? And I think the key example, the classic example for this is a scenario where you have custom forms, where you have an arbitrary set of forms that are defined within a database by a form object. And then you can fill those forms in attaching arbitrary data, perhaps in my case using Postgres, where you can then have a JSON column to just contain a hash that would represent the filled form. And in that case, actually the set of validation rules will be specific for each form. And what we want to do is be able to define that in a way that will run on both the client and the server. So perhaps actually the problem I'm trying to solve is not necessarily the problem in the simple case, but the problem in the harder case. But what I hope to reach by the end is a suggestion that maybe there's a way to also solve the first case once you've done the second. So this is the basic example of the custom form idea. So we have a model for a form, which has a title, and then it has a definition, for which in this case I've made a new transform called JSON, and that will contain a JSON hash which describes in whatever means you describe the structure of a form and the structure of a form input elements and things like that. And then we can have filled forms, which are the forms as filled in by the user. So every filled form will be linked to a form. It may have a user or any other set of attributes, but crucially it will have a form data hash. And so for every form element that's defined in the model, we would then define what the user has inputted into that form, in the filled form. Now, have I explained that clearly? Good, OK. I'm seeing some knots. Excellent. Now this is the transform that I use for JSON. I think I nicked it when I randomly searched on Google. I think the simplest one I've seen literally just sort of passed through the value unchanged either way. This one, I think, does a little bit of munging for reasons that I never really paid much attention to, but it seemed to work and it's never caused me problems. And then this is an example of a form. So I've already told you that I'm medical by background. So this was actually thinking about forms for referral forms for some sort of medical record system where we could have a form that's for an inpatient physiotherapy referral. And I've made up my own definition structure here where a definition is an array of hashes, each of which defines an element type, the key for that element, what input type is going to represent it and the label. And then I've got code which I'm not going to go into, which then helps turn that into a form to display and to fill in. So I think that the general specification for this problem is that, or the way I'd like to go about it, is not to have to write new validation tools and simply to couple existing validation tools to avoid as much extra work as possible. And the whole thing has to be sort of completely run time. So it can't involve, for example, rails generating some ember code for you to put in. And then every time you change the forms, regenerating that and updating your client-side code. So that then gave me, I think, three main parts to the problem. The first part was that actually now, instead of declaring my validation details somewhere in code, I now simply want to store them and serve them up. And then I need to allow validation in rails and validation in ember to be dependent upon the stored validation rules. So the handy thing here is that the format for ember validation is already JSON, so it seemed a completely straightforward step to use the ember validation format as my storage format. And then in order to allow data-driven validations in rails, I had to think about this and I looked and I'm not good enough at Ruby to sort of open up the internals of active record validations and work out how I could sort of automatically call the right things to set these up at runtime other than by working at how to do metaprogramming, which seemed to be a bit easier, where you actually alter the definition of a class at runtime and a bit of trial and error made that work. And then similarly, we need to allow data-driven validations in ember, so for which I assumed I would simply need instead of, as you saw before, declaring the validations hash statically in the code, simply loading the validation hash onto the controller or the component at runtime. Oops, I went too far. So this is an example now of a form to which I've now added a validations hash in the database and the validations hash, as it looked like before, defines a key and then has two criteria here. I mean, those two are a bit pointless because if the length... Oh, no, no, no, yeah, so you have to fill it in and you have to have a length of at least five. So on the rails side, this was how I ended up doing it. It took, like most of my programming, it took quite a lot of searching on Google and looking at people who know about how to do Ruby to get it to work. And I'm going to go through it bit by bit because it's not that well commented and otherwise I would just be standing here and you would read it and that would be a bit boring. So a filled form belongs to a form. That's what tells us the structure the content should take. And instead of declaring any validations directly, we simply say once a filled form object is initialised on the server, we're going to set up the validations. And it's not a rails evening, so we're not going to focus on this too much. But we then load the form definition and load the form validations. Now the next problem I encountered is that active record validations, as far as I could tell, it wasn't very easy to validate something that was a hash attached to your object. It would validate first level properties and not necessarily other things attached to that but at least it didn't seem straightforward. So what I first needed to do was actually proxy the form data hash. So instead of the filled form having a hash, which was a key value store of all the things, I wanted to translate that into a series of direct properties on the object. So essentially for each of the form keys, I define a singleton method which will have the name form underscore and then the key. So that gives me a slightly namespace safe way of doing that. So by the end of this little do loop, I have now translated all of the elements of the form that were defined through a hash into direct properties prefixed by form underscore on the filled form. And now I can trick rails validations or active record validations into validating these. And in this case, it took me a while to work. I still can't work out quite why it is that one. I thought it would be instance eval as the way of sort of applying something to the class. But it turned out it was class eval. And this essentially reopens this instance of the class alone. And then for each key, I simply add a validation rule converting the namespace version of the form key to a symbol. And then deeply symbolising the deeply symbol... I should stand further back. I can't point it out. I'm casting shadows. Deeply symbolising the hash that describes the validation. So you can see there. So reason with presence true and length minimum five simply gets turned into the command below validates form reason, presence true and length minimum five. I still don't know if that was a sensible way to do it, but it seems to work. And it didn't involve me writing much code. And I'll look forward to any critiques later. But does it make vague sense? Good. That's also more nodding. Excellent. It's vaguely interactive now. I'm getting nodding audience feedback. So on the emberside, which we're more interested in here, I thought it would be really straightforward. I thought, oh, at the time of initialising the model, all I need to do is sort of load the validation's hash and it will all work. It wasn't quite that simple. So this is the form model. So now I've added a new parameter to the form's model called validations. The first thing to realise is that that contains the validations with the raw names of the keys. So the name is reason in this case. But actually, on the filled form object, similarly to the rail side, actually the reason property is actually on the form data hash. So it would be addressed as form data dot reason. Now ember validations actually handles addressed variables, very easily, so you can validate things that aren't sort of simple properties of the direct object in question. So validating a sort of form data dot reason, a key on a hash is absolutely fine. So what we then do is have a dependent, a computed variable from the validations, which essentially takes each of the keys in validations and prepends form data dot, so that we now have a new validations hash defined on the basis of the first one, which refers to the data as it will be seen in ember. So this was my first failing attempt, which was why not save time and just use a binding. So the component that I have here, which I've just named here fill custom form, mixes in ember validations, and this will be given the property of the form that you're filling in as the property form through the component syntax. So I thought I could just do this. I could say validations binding and bind it to the client validations on the attached form. That didn't work, probably because of the way that ember validations accesses the validations hash at runtime, but I don't understand. So I tried again. This time I said, OK, well, let's just do it manually. So on initialising the component, what we'll do is we'll just statically set validations to the computed client validations from the form. That also didn't work, and I suspect it may be because my initialise doesn't get there quite quickly enough again for reasons that I wouldn't understand because I don't understand ember internals well enough. So then I thought, well, can we do it another way? And it turned out it worked. It's a bit hacky, and it takes things into the pandelbars templates that I rather not, but what I've done actually is the fill custom form component when I include it. I give it the form. I give it the filled form. I also cheat, and I actually directly give it the form validations for whatever reason that seems to work. But perhaps it would be better for me actually to spend a bit more time understanding the internals of ember validations, and then we might work out a better way to hijack it at runtime. So that's it really. That's the one achievement I think I've managed is some sort of almost polymorphic data driven and type specific validation for objects. I don't know if that's too niche to be of use to anyone else, but I think people having data defined objects is not that uncommon, but the limitations. So at the moment it's got no capacity for handling non polymorphic validations, either uniqueness, because it's not really possible to do it on the client, or ones that you may only wish to do on the server or the client for valid reasons. I haven't really gone at all into how you would then deal with errors, merging errors on the client and the server. In a sense it should be quite straightforward, because most of the time, if you get a client error you won't submit, if you don't get a client error you will submit, and if your client's not trying to trick you by hacking into you, then it should validate on the server side unless it's a uniqueness constraint or something like that. But maybe there are issues there that need to be solved. The slight complexity there is that the Ember validations errors come on the component you're programming, but the server side errors actually come directly on to the model, so they're stored in different places. Custom validations would be a whole another barrel of fun, because then you're actually having to write executable code, so you'd have to either, you'd potentially have to define your custom validations once on both sides before you could use them, but perhaps there's other ways. It was suggested to me when I chatted to someone here previously that there are efficiency problems with metaprogramming in Ruby, so whether this would have problems with scaling, I don't know. Where could I go with this? I think there could be ways of trying to overcome the limitations, working out ways of defining custom validations in both JavaScript. One thought, and this would probably be an awful one, but if you could define them in JavaScript only, serve them over the wire and run them on the server side with XXJS, but I think that would probably be horrific, I don't know. But then going back to the first problem, which is the fact that actually even in the simple case, we still have to declare our validations on the server and the client side, can we take this solution and backport it onto standard validations so that we only have to define our validations in one place? And if we were to do that, I think the neatest way would actually be to move away from storing them in whether in the code or whatever is some sort of JSON hash, but actually defining them through the active record validations and then having a single validation root on a server, for which then you could query for any given model and get the validations hash for that. But that would require some way of interrogating a model to find out what validations have been applied to it. Now, I'm sure there's a way to do that. It would just require looking it up, but again, is that a good idea or not? I don't know. But anything that saves me writing more lines of code in the future I usually think of as a good thing. And if it turns out from this that actually it isn't a terrible idea and this isn't an awful way to do it, then maybe it would be a sensible thing to share, but we'll see what you think. And I won't be upset at all if you tell me it's a mad idea. So thank you very much for your time and your patience. I hope it made vague sense. I hope it was slightly interesting. Thank you. The answers probably don't know. Yes. I don't work on the other side. So is it actually, do you ask the function or is it a data problem? So I guess if I go back to really near the beginning, I mean, again, I confess I haven't used it much. This was a problem I came up with fairly early in my course of using them. So essentially you apply it to a component and then you tell it what to validate and how to validate it. And then it gives you the methods to do that validation so you can then check is this valid or not and then make logical decisions about what to do with it if someone presses submit in that case. It will also validate content live and so it will update a hash of errors live as you enter into a form, meaning that you can then display areas live. I mean, I've got, if I take it off full screen, the system that I'm sort of doing in my spare time, the example here is an inpatient referral for which we choose physiotherapy inpatients. You can see actually before I've even started typing, it's validated and said it can't be blank and it's too short. So I mean, again, which is something you usually want to change in practice because you don't want to validate immediately and it will update that validation live. I've tricked this. So actually if you press submit, it will genuinely submit it and then the server comes back and says the same, but I think I'm misinterpreting the server hash, which is why it appears on both things. So you can see it repeats the error twice, which is why I did say that I haven't solved the problem of dealing with both errors at once. So it's quite a nifty and straightforward way of doing validations. And then of course once you link it with all your CSS and everything, you can make everything look whizzy and nice like the people at British Gas did very quickly. I don't know if anyone's there for that, that was good. So can I continue? Of course, sorry, yes. Okay, we'll come back. Did you consider building something like a build step to say Jason and then generates a jealous build and will be called like a watch customer? Yeah, so that crossed my mind, but I think going back to sort of earlier in the talk, if this was just for statically typed models, that would work perfectly. Yes, some sort of build step that then means that you define it once in whatever way and you then produce the static Ruby and JavaScript code for that, which is then part of your project. I think that would be absolutely fine. In this case, because it was first, my first problem was actually with a model that has different validation rules dependent on which form you're filling in. And those could be defined at runtime by users so there would be nothing to stop in this system a user defining a new type of form. So actually it had to be stored in the database and it has to be turned into executable code at runtime. But yeah, for the simple case, that's something like that would work. Just trying to understand. So you were saying that it's, you would have to, are you trying to avoid the case of serving up code and then having to update that code again. Yes, so. Can you not have an API where you get that validation object? So I think, yeah, so coming back, so I guess it's just keeping the two problem domain separate. The problem domain that I actually went in depth into here was a sort of a per object validation where every filled form will have a different set of validation rules depending on the form that's being filled in and that those are defined in a database and they can change. So that actually takes us a little bit beyond the usual problem of simply having our model for a user which has asset of rules which you want to apply on the client and on the server all the time. So this is saying, actually, you have dynamic validation rules. But then what I tried to bring in at the end was the idea of, yes, backporting this to the more simple case, in which case one solution would be, yes, you simply do it all statically at compile time and you simply compile your validation rules. I thought you were talking to set of rules. One rule is for the data object and other rule is for form. So, for example, you define five object types like a patient, a doctor, and now, is there any defined 10 forms? Yeah. In the database, yeah. And still it's a static. It's not a static. Maybe we... I don't know if we differ slightly in the way we're defining static and dynamic, but I think for me it's dynamic because while the server is running and while the client is running, if I change in the database, the validation rule, so this is served up from the database. Yes, this is form number two, served up through a request to the server. So if I change this on the database and the client requests from reform two and I've changed it, it will get a new set of validation rules and those will then be applied when the form is filled in. So I mean dynamic in that sense as it's not part of the static code of the website. It's one thing that occurs to me. I've done a bit of Django. Okay. One of the interesting things about the way Django is laid out is that forms are a separate kind of domain concept in the framework and models. Right. So you can define in that sense a form which may well at the end of the, you know, generate a model on the server side, but it doesn't have to, and that doesn't correspond exactly to the model. And you could... I'm not sure it depends... It's been a while, but... you could maybe customise these things at a long time. And if anything, that general approach would be easier to... I see. Yes. ...of these reasons. The other thing is, I mean, it just occurs to me, you could think about it like something like that, having a kind of factory object over it, you give it some validation rules and then that does the mad meta-programming on your room B class, really active record model... Yes. ...and returns you a custom sort of singleton class of that active record model. Right. Then you could use that to do the validation rather than having it all in a big method. Yes. Whether that would actually play nicely with all the other magic that I'm going to have to do. No. I mean, I guess the first easy step is simply to take that function out into a... I forget what the version of it is. What do you call a mixing in Ruby? I don't know. It's called a mixing. There you go. To take it out into a library so that you know, because at the moment, this would just be copied and pasted across modules. Yes. To be loaded dynamically in Ember. Yes. Do you try using it in computer property populations? I don't know. Do you mean that... Because in this case, I did, I guess, validation's binding. So that should set up... I don't know. I don't know. Right. Right. And is that distinct to doing a binding? Yes. Okay. Fine. Right. Right. Oh, maybe that would have worked. Right. Yes. And if not, it's about five lines of code to make it promising. Is it? So I would almost completely... I guess, almost completely surely that it's probably not promising where, because all of the examples I've seen of Ember validations are all about using it statically. But, yeah, that might be... I'll take a few comments. Okay. Great. Oh, well, that would make things much better rather than putting it in the template. Do you want... No? All right. Yes. So two things like a common pattern in Rails is to use view doctors. Right. To get all, like, the form logic in there or to have a separate class that inherits an active model. Right. It behaves a bit like a model, but it's not an active record model. Right. But the question I want to ask is, where do you actually define the validations then? In the database. So did you have, like, an admin tool or did you just, like, stick them in the seeds file and then... No. So this is just... So each form is a row in the forms table. And it has an ID. It has a title. It has a definition, which this is postgres, so I can have a column that's a JSON object. So the form is defined, has a structure that's defined on the... Well, no, because it's just data. So it's not changing the structure of the database. So to change a form's validation rules is simply to change a field in the database. To add a new form is just to add a new row to the database. But I was interested in the mention of active record views, because my imagination is that because that's when you're using Rails as you're serving up the web pages for more sort of classical work. What I thought that I'm a doctor's folder that I'm a doctor's... Yes, I see. ...and the active record model, and all the logic type of the form is just on a view doctor rather than that. I see. So my suspicion is that perhaps might not apply so much in this case because Ember is doing... because so much of the form stuff is taken over. Yeah, yeah. Great. Oh, well, thank you very much for your time. One more question. Oh, one more question, yes. When do you find time to do all this? Oh, well, the last job... I haven't touched this for two months, which is why all of today was spent reminding myself of what I'd done, because my last job was not so busy. Actually, it was fairly... it was good. It didn't stress me out too much. It wasn't too many hours, so I actually did have a lot of time sitting on the sofa doing programming. For the last two months working at St Mary's, that hasn't happened at all. Well, thanks again, Andrew. Thank you.