 We're going to talk about core data in Ruby Motion. My name is Lori Olson. I'm a freelance software developer, trainer, and mentor. I'm from Canada. I just moved to actually, I used to live in Calgary, and I moved out of Calgary because I work remotely most of the time now, so I don't need to live in a big city anymore. I would like to quickly thank my current client though for kindly letting me sneak away for a week to attend this conference. Kudos, they do employee engagement and reward systems, so. I'm a developer, mostly Ruby, mostly Rails. I do my Ruby Motion stuff on the side for personal projects mostly. I did a little bit of JRuby. I've done a lot of JavaScript, jQuery, a lot of mapping and charting systems, and I've implemented JavaScript testing in far too many development teams that should already have been doing it. I also teach a couple of workshops. Ruby on Rails for real developers, which is focused on developers like Java developers or .NET developers, they're already familiar with the basics, but just need to get up to speed on Ruby on Rails. I also teach Rails for iOS developers class, which you guys probably don't need, but the stock Objective-C guys don't know anything about Ruby or Rails, so it's a handy course for them. I also volunteer with an organization in Canada called Ladies Learning Code, and I actually teach their Ruby introductory course to women who want to learn how to be developers. I also speak at far too many conferences, and I got to cut it down because it takes up too much time. So anyway, we'll start off with, this is a bit of a story. I wanted to actually learn Ruby Motion. When I set out here, I had tried and failed to get going on iOS development using Objective-C, just because I really hated Objective-C. I found it difficult, and every time I switch back to working on my Rails projects, switching back over to iOS and Objective-C just was painful, and I hated it, so I never actually finished any projects. So I always try and pick something I'm already familiar with when I'm trying to learn a new technology, just so I'm not trying to do too many new things at once. So I wanted to take an existing mobile web application that I had previously written and redo it as an iOS application. This particular application is called Welles in my Backyard. I'll just give you a brief overview so you understand the problems involved. The story goes, it's about four years ago or so, there's this little community in Canada in Alberta called Kalmar, and in a new development in this little community, there was a lady who thought she smelled something funny in her backyard, and it turned out she did smell something funny in her backyard because she had an improperly abandoned natural gas well one yard away from her back door. I thought at the time this story broke that that was absolutely appalling that you could have a well in your backyard and not know it. And since I actually was working for a company that had all of the oil and gas data there is in Canada, I thought, I can do something about that. And operating under the assumption that it's easier to get forgiveness than permission, I just extracted out all the abandoned well data and used it for my application. So there's a few challenges for this particular application. It's lots of data, and it needs to display on a map because otherwise people won't sort of understand location data. I also need to display this data in a list. And I need to do filtering by location so that people can see overviews or specific, you know, their backyard or their community, stuff like that. And I figured how hard could this be as an iOS application? I'd already done this as a Rails app, sure. So this was a screenshot from my data export of the abandoned wells. There were, I don't know if you can read that, there were 244,292 abandoned wells in Canada. That's just down from the over 750,000 oil and gas wells there actually are in the country. And it occurred to me that, you know, being a Ruby developer, there's probably a gem. I can use a gem. And I'll be able to load up all this data. There was Nitron, which seemed to be abandoned code. There was motion data wrapper. There's motion model, motion migrate, magic record. There's actually too many more to list. There were so many gems. And I kind of went through them all. Okay, that's the subject of another whole talk, is the amount of time I wasted going through them all. But the problems that I ran into were that the gems and DSLs that people have written hide a lot of what's going on in core data, behind a lot of magic and obfuscation. And the straight up iOS Objective C development relies a lot too much on the Xcode magic. And it turns out that when you have complex data, it's just complex. And large data is large. And sometimes these solutions, cookie cutter solutions that the gems provide just aren't good enough. And it turned out in my case, I couldn't actually get any of the gems to do what I wanted them to do. So sometimes you need to step back from all the gems and just figure out what the most base level APIs are that you can use and figure out how to use them. Now it turns out Ray Wenderlich has an awesome set of tutorials on core data. But this is of course all in Objective C code. So I went there and just started reading it. You know, my hate of Objective C aside, this stuff actually was fairly decent and I learned a lot from it. And it turns out that once you actually start going through these examples and you go to the Apple documentation about core data, it's not really as scary as it looks if you just sit down and read it. It's actually pretty straightforward and not rocket science. So the problems I was trying to solve are creating my models, entities in code, not in Xcode with its magic modeling tool. I also needed to define relationships between those models in code. I needed to load up that data to 144,000 wells and because it's so much data, I needed to do some work around optimizing it. So models in code. There is a sample for that. If you look in the RubyMotion samples, there is a sample app called Locations and it is a core data app. So if you haven't looked at it and you didn't know that, you should go and look at it. It follows the MVCS pattern, so model view controller store pattern. So I know Colin will be happy about that whole store thing. But I think it was the big nerd ranch guys that came up with that pattern and it's quite useful. So let's just go through it. This is the location model. That's a bit of a wall of code, so let's just break it down into its component parts. First we define an entity in code. And it's just defining in any description, giving it a name and telling it what class is its managed object class. It's pretty straightforward. The next step in there is defining its properties, its attributes. And in this case, there's a little bit of Ruby magic going on here, but we're taking pairs of attribute names and types and throwing them into a loop and creating our NS attribute descriptions. Using those, defining a property name, defining the attribute type. And in this case, we're just hard coding that these properties are optional. You could pass in a third parameter here and choose whether or not these things are optional or not. So we've now defined all our entity properties. It's not too bad. Then we need to actually set up all the stuff that... And this is something that you need to do in Objective C, by the way. Set up all the stuff that your iOS application needs to use a core data store. So this is in our location store file. And we'll break it down because it's not really as difficult as it looks. First we need to set up a managed object model and tell the model about our entities. So in this case, we have an array of one entity. There's only one, the location model. And then we need to set up a persistent store coordinator. And that needs to be initialized using the managed object model that we already just created. In this case, we're going to use a file in a directory that is writable because this application needs to write to it. So the URL is going to be in our Documents path and a file name is going to be called locations.sqlite. And then we get the store to add a persistent store of type sqlite. So again, this is a fairly straightforward process here. It's just step-by-step managed object model, persistent store coordinator, managed object context. That's the third piece. And again, this is not difficult code. It's just creating the managed object context and telling it about the persistent store coordinator. And that was easy because that's all there is to using core data in the location data. So if you guys bring that app up and look at it and create locations, you will be using core data and creating core data objects. But unfortunately, it turns out that's a little too easy because that's where I came to a grinding halt in my little story trying to get things going with core data because I didn't have one model, I had two models. Two models with relationships between them. And there's reasons for that. When you are displaying things like on a map, you need stuff like information, textual description for something that's going to be on the map, and you need location coordinates. That's pretty much all you need. But when you want to display, say, the details about those things that are on the map, then you need a lot more information and you probably don't need the location at that point because you already saw where it was on the map. So in this case, I had two different models and I wanted them joined together in a relationship. And that's where that whole thing breaks down because when you set up the entity and define its properties, you need a reference to another entity to define a relationship. So you're creating an entity, so you're creating your first entity, and then you go to define the relationship to the second entity, but you don't have that object yet. It hasn't been created yet. So how do you do it? Yes, it is the classic chicken and egg problem. You can't set up one without the other. But there is a solution. Defining relationships in code doesn't have to be difficult. I know in the Xcode data modeler, you just wire everything up with pictures and it's all pretty and nice and it just works. But you can actually do this in code and it's not that difficult because what you do is you define your entities first, define all the entities first. Don't try to define the properties yet. And an entity's properties are actually a combination of its attributes and relationships. So you define those lazily. Later, after all the entities have already been defined. So again, we'll go back now to locations, which I'm starting to modify now into something a little bit more like Ray Wendellic's example, which is his failed bank scenario. He has failed bank info object and failed bank details object. These are two objects and there's a relationship between them. That was pretty much exactly what I wanted to do, so I figured I'd stick with this as an example. So if we just define the entity, I was able to clean things up quite a bit and now we just have an entity being created, a name for it and I'm just giving it the class name and the manage object class name is just the class name and wait a minute, this code looks like you could put it anywhere because it doesn't have to be in my model because there's nothing real specific except for the name. So this is something that you could start abstracting it into a super class. This is probably where most of the core data gems start on stuff like this. I didn't bother, my example just leaves it like this and all of my models have this in it, but be smart and don't duplicate code. So our next step is back in our store class, which now is looking like a bank store, not a location store anymore. We're now creating a manage object model. We are setting up its entities, which is just taking those two classes and calling entity on them. So that gives me my entities and now that all the entities have been defined, I can go and define properties on them. So for each entity, we're going to set up the entity properties. So this is my set entity properties and this is just the first part of it where we are going to set up the attributes. And we're pulling out the manage object class just so that we can access its class level variables like attributes and relationships, which are defined in the class as well. We'll get there. So for the manage object class attributes, it's going to return an array of attributes and for each of them, I'm just going to define a new attribute. And it's got a name, a type, a default value and whether or not it's optional. If we look at, go back to my model class, this is the attributes class level method and it's just an array of these hashes with name, type, default and optional. This is pretty straightforward Ruby programming. This would be really, really ugly to do in Objective C, by the way. Okay, back into our property setup. Now we're going to do the relationships and same as the attributes, there's a class level method called relationships that's going to return an array, well in this case it's an array of one, but it's going to return an array and we're going to define an NS relationship description. And I know it looks like a lot of stuff, but it's all very straightforward. We define the name for the relationship and then we get the destination entity, which we have. We have this entities by name thing here from the managed object model. So it just gave me an array of all the entities by name there. So I can use that to look up all of the entities. So I can just pull out destination entity there and similarly I can define the inverse relationship because you should always do that guys. Even though if you don't think you're going to use it, just do it, it's a good idea. And it gives some hints to core data for doing queries and stuff. And again we have optional property, transient, whether or not it should be indexed, whether or not it should be ordered. The min count and max count is the arity, so is it zero to many, one to one, one to many, zero to many, whatever. That's what your min count and max count give you. And then the delete rule, which basically says, if I'm going to delete my object, what should I do with this thing that's attached to me? So I've given defaults, so you don't actually, in this case, have to define all these things if you don't want to in your little hash that's going to define these relationships. But you do need the name, the destination entity, and the inverse relationship. So this is back to our model again where I've defined this one single relationship in an array. So this is our failed bank info object, and we have a relationship to the details object. And the destination is the failed bank details class. And the inverse on that class pointing back to us is going to be called info. It's optional, transient, false, indexed, false, true, and this is going to be a one-to-one relationship because we're always going to have a detail for every info. And we want it cascade deleteable. So if I delete my info record, the details record will also be deleted. And the last line is setting up our entities properties, which is simply the attributes and the relationships. They're both arrays. We just add them together, throw them in, and we're done. This is just sort of more of the same back on the details record. I'm not going to break it down, but we have attributes for it, and we have the relationship back to the info record. There's nothing different here from what you've already seen. So that is all you need in code to define models and to define relationships between models using core data. So with this, you can define pretty much any kind of model relationship that you want to, and you can do it all in code, and you don't have to use XCode's data modeler to do it. I don't actually think this is that complex, so you ought to be able to do it without using XCode. My next step was that data learning problem. So I want to get all my well loaded up into my data store. So once again, I go back to raise wonderful tutorials because they are wonderful, but of course, all of his stuff is object-to-see, so I had to massage and translate it into movie motion code. So I wrote in my store class, I wrote a load function. We'll break it down step by step. Again, this is pretty much a direct translation of Ray's example, so we're using his file, which is the banks.json file, and I just threw that into my resources directory, so we can pull that out. I'm using bubble wrap here, because it has some nice JSON parsing built in, so we just parse that file, and it throws it into a banks array. So next step, we have a banks array, but I want to put all this stuff into coordinates. So I get my singleton from the failed bank store class. That's something from the location example. Go and look at the location example for singleton. And I call add bank. This is just the direct translation of the add location that used to be there, but now we have two entities that we're going to be creating. The info and the details entity. And we're just going to populate them with stuff that came in from the JSON file. So in this case, it was the bank's name, city and state. We're going to do the update date as today's date, because I'm reading it in, so why not. The close date, that's a really nice little function date with natural language string that just translates dates for you from strings. And the zip code. So all of these are populating the two different objects, the info object and the details object, and then we simply assign the details object to the details relationship on the info object. But wait, what did that add bank method actually do? All it does is it yields to the caller. It's expecting a block, yields to the caller, two newly created entities. So we're creating an info object, any details object here, and passing them into that block. And after that block, they're all properly populated. It comes back here, and it calls save, which saves the data into the store. So again, that was pretty easy. I thought it was pretty easy. The bank's JSON file is a nice small file. So, yeah, so a small file is great when you're doing something like this, read in the whole file. Like seriously, how many wells am I reading in? I have a file that has 244,292 wells in it. That ain't gonna fly. So we're not gonna read in the whole file because that's a problem. That's a fairly serious problem. It's just gonna blow up when it tries to read that in because there's no memory. And this, by the way, is a problem, too. So we had this add bank, and we iterated through this array of banks. And I was just like, create a bank, save it to the database. Create a bank, save it to the database. Create a bank. Okay, if you're familiar with doing data loads, this isn't a very efficient model. You don't want to do a save for every add. What you should do is save that up, but you don't want to save it all once for the end because once again, 244,000. You're gonna run out of memory again. So that won't work either. So what we really wanna do is batch it. So you wanna get some sort of reasonable number of objects created and then save it as a batch and then do another reasonable number of objects created and then save those. Which seemed like an awesome idea, but I still wound up running out of memory when I tried to run this in the simulator or on my phone. I was getting really frustrated, but hey, Ray figured this out too. He actually had a really convoluted way of doing this in his tutorial, and then he went back and ripped that out and changed it over into something that actually works well in RubyMotion too because you can just use an OS X app and use all the same stuff because the core data classes operate exactly the same way on OS X. So I just created an OS X app using RubyMotion, copied all my models in, and I did it that way. So we have a new and improved load. We'll break it down again. This time I don't actually have a JSON file. I have a CSV file because I was exporting it out and that was what my database export gave me, and we're going to actually keep a count of the load because if you aren't doing this and you're doing a large load and it fails in the middle, you don't know where it went or how long it took before it failed. I also hacked up a version of faster CSV to work. I understand somebody else actually did a proper version of this, but if you look at my source code, this is a hacked up version of faster CSV that I'm using. And it streams the file. It's just reading parts of it at a time so you don't load the whole thing into memory. So one line at a time out of the CSV file. Now instead of calling add bank, we're calling create bank. Guess what the difference is? It doesn't save every time. So and now I'm using my real data, but I'm just going to stick with the failed bank thing, so I'm just stuffing values in wherever the types match up so I can see that it works. The next step is, okay, so we created a bank, no, a well, so we increment our load count and now for every 100, I want to save. So that seemed like a reasonable number. Every 100 I'm going to save to the database. And just because watching it run and run and run and not knowing where you are was really boring for me. I started printing out a dot for every time that it was doing a save. For 244,000, that's a lot of dots, so every 1,000 I printed out a star, so it actually gave me a nice indication of progress here. So saves every 100, progress at every 100 and every 1,000, there's a different progress indicator. That's a nice thing, and when it's running for 20 or 30 minutes, it's nice to watch. So again, at the end, once you come out of the loop, you've done all this, chances are you didn't end on an even 100, so you're going to want to do one more save to get that last uneven bit out of the way and then print out how many you loaded just so that you know you actually got them all, and that one actually worked. So I finally had all of my wells loaded, and then my table of each hope because it couldn't display 244,000 items. So optimization. Hey, ready to the rescue again? He has a NSFetchedResultsController example. So we did NSFetchedResultsController because it gives us benefits, and it's made to work with core data. This is one of the reasons why I actually chose core data is because I had so much data and I knew I would eventually need to use an NSFetchedResultsController. It basically just avoids the problem of loading all that data at once. This is literally a direct translation from the objective C, so it's a little ugly, and again, some of those gems out there help you with that ugliness. Much greater fetched results controller. All you need to do is create a fetch request, tell it what entity you're working with, give it a sort descriptor, and give that fetch request to your newly created NSFetchedResultsController. Done. You do need now to use this in your TableViewController. So in my ViewDidLoad, I create the fetchedResultsController, and I set its delegate to myself, which is the TableViewController, and at the ViewDidUnload, I get rid of that so I don't have a memory leak. Yeah, so there's a couple... Again, this is the... Guys, use Promotion, really, but PromotionGuys, how do I use NSFetchedResultsController with it? Okay. Yeah, so breaking that down, there's at least three things you need to do here. You need to implement your TableView number of rows in section, and you just get that from your fetch controller. You need to have your TableViewCelferRowItIndexPath, and in this case, I have a subsidiary method here to actually set up the cell, which is, again, just getting the object from the fetch controller, and now I have my entity, which is my bank slash well. I use the name, city, state, boom. I've configured my TableViewController to use NSFetchedResults. There's one more gotcha. There's the delegate. The NSFetchedResultsControllerDelegate. Remember I set delegate to cell? That's because I have this included module. This is... I don't like this slide. This is big, scary wall of text, because you guys will just copy this, because that's what Ray said. He just copied his out of the Apple documentation. So that's what I did. I copied Ray's, and I translated it all into RubyMotion, and I recommend that's all you ever do is copy this stuff around. Unless, of course, you're doing something tricky with editing, you may have to change, like, one or two lines in here. So that was pretty much the end, and I got my app to work, and I got all my data to load, and it came up in a TableView, and it came up in a map, and I really wish I had time to do a demo, but I'm already running overtime. So we created models in code, we created relationships in code, we preloaded a lot of data, and we now know how to efficiently display it. So that's the basics of core data, and you can do it all in code, and it's pretty straightforward, step by step by step. There's also lots of great gems out there, so look at them, but now you know how it all works under the covers. So if you ran into problems like I did, you'll know enough to go in and make changes that I didn't know how to do at the time. So this is my dog, Finnegan. He thinks you should all use core data now, okay? So I did actually find myself with enough material to write a book, so I started writing a book. It's not quite finished, it's almost finished, it's about 80% done. It's up, and you can go to coredaydayinmotion.com. So does anybody have any questions?