 Hi, everyone. Today I'll be speaking about building an ORM with Irid. My name is Vipul. I'm supposed to be presenting this along with Prathamesh, but apparently he wanted to go to RubyKaigi. He's presenting there, and he's having fun with lots of food and nice Japanese cuisine. I work at a software consultancy based out of Miami. I work remotely. It's mostly Ruby on Rails, building web apps and things like that. As you mentioned, I run Ruby India back in India. It's a platform for helping people, helping, like, lots of Ruby people get together, like, host their content or host your blogs and all those things. I do a newsletter which is named as Ruby India newsletter. You can check it out on rubyindia.org. We've recently started with podcasts at rubyindia.org, where we do a lot of podcasts from a lot of people from the Ruby Indian community. Also, we are going for the past month I've been traveling, so I got a chance to interview people like Koichi Sasara or Fabio Akita and such people. So you can go ahead and check it out. A bit about Ruby Indian conferences. So there are lots of Ruby Indian conferences that are coming up, main among them is Ruby Conf India. How many of you have heard of Goa? So if you like Goa, you should attend Ruby Conf India. You get to stay at a resort. There's Garden City Ruby Conf which is in Bangalore and Deccan Ruby Conf which was held two months ago in my hometown, Vijaspune. So if you'd like to visit India, do come around in one of these events. Before we begin, it's been ten years for Rails, so maybe a round of applause for that. We had this awesome event where we celebrated launch of Rails. And this is one of the issues which was by Hiro Osari on Rails issue tracker. So if you get a chance, go around this issue which is 16.731 and do show your love for Rails. Anyone who's from the Rails team over here? No. Anyway. So yeah, as you mentioned, when in San Francisco, wear your own plain t-shirts, I was wearing a brainy t-shirt which I got in Madison and they were like, yeah, anyone from Braintree? Okay. So I got people asking me, like, five or ten people asking me, oh, where's your office? Or do you know Jack? Oh, is your office in Massachusetts also? And, like, it was really embarrassing. So wear your own t-shirts. Coming back to the presentation, I'll be walking through. It will mostly be a code walkthrough. We'll be seeing a lot of hands-on code. So please bear with me. There's a lot of code to see, so I'll be a bit fast. Before that, we'll need to take a look at how do we create caches, but he has already covered it. So why build our own ORM? Basically, we'll try to experiment and build a tiny ORM, which I'm calling a storm. We'll try to take a look into what are the different aspects involved in an ORM and get a bit better understanding of what different things go around in active records so that we get a better idea of what we are dealing with. And if we fall into, like, different bugs or we fall into hurdles, we get a better understanding of whatever we are working with active records. So what is ORM? ORM is in scientific terms. It's something which is used for converting into incompatible types. We can call it as a virtual object database, so, like, from a database to helping it out for accessing in your native language, like, in our case, it would be Ruby. There are two basic patterns for implementations for ORMs which are used, which are implemented in Ruby, one being the data mapper. We'll be taking a look at active record today. So what active record is going to wrap a row in a table and encapsulate accessing of that particular row, giving you methods like instance methods for accessing information on that particular row. And on top of that, you can define your own domain logic for what needs to be done on that particular top of that particular row. Different components we'll be taking a look at is database connection, how do we actually connect with the database? We'll be taking a look at Postgres today. Query generation, which will be where Arial will be helping us to actually generate the whole queries, attribute accesses, which we'll be taking a look at and type casting mechanisms. So what we essentially want is if we have this kind of table structure where we have post model, sorry, post table where we have ID, name, content, authors, and so forth, and we have this information over here, we essentially want it to be represented in the form of a basic object in Ruby, which has ID, name, content, author, and so on and so forth. Behind the scenes, what will be happening from our ORM is it will be converting all of these single fields into integer strings based on whatever value is being written to us from the database. So what we want to achieve is something like this, what you can do an active record, post equals to post dot new, pass a hash off values and you can do post dot save, you can do post dot find and so on and so forth. How we are going to do is we are defining our own ORM over here, which is storm, tiny ORM. I'll be giving links later for checking out the actual code and if you inherit from this particular model, you'll be able to perform different operations similar to how you do it in active record. What is the role of Arial in whole of this? We are going to use it for actual query generation. A bit about Arial, it's an acronym for relational algebra. It is mainly used for, it is a SQL EST manager, like all of the different things that are involved in a query, they are being, like the different things which are involved in the query, like select or all of these things, they form, they are formed as an EST and Arial is helping us to create that, traverse this particular query. It can generate a lot of complex queries like unions, joins and other things and it adapts to multiple databases. You don't need to worry about in case of Postgres or in case of MySQL, how does the limit clause work or how do different things like Windows work in different databases? It works on its own. These are the components, basic components of Arial's EST. As I said, it is used for actual having information of parts of the query and it works in terms of nodes. So every single node has one basic information of how to work on, let's say, where or limit and so on and so forth. We have visitors which are going to traverse through all those, that particular EST and work on that particular nodes and find, convert it into strings for execution. And we have managers, visitors don't know how to traverse a particular tree. So managers are going to help the visitors to actually find out, like, in case of insert, we have insert manager, in case of insert you have to go this way, in case of update you have to do updates, in case of delete you have to delete. So this is essentially what we are going to implement today. We have the O and M sitting in between of database and Arial and RORM will be allowing access, will be exposing access to the database to the user. So Arial will be used for generation of queries, which are sent to database. So the first part that we need to implement is basic engine, which is going to handle, which is going to provide information to Arial regarding the tabular structure, the column structure. How is the table? What kind of database does the table have? It is going to provide database adapters. So in case of active record, currently we have SQLite, or we have MySQL, MySQL 2, or Postgres. This is being provided by database adapters so as to, so as the, like, Arial knows what kind of coding mechanism or what kind of actual processing of coding should be happened by the database adapters and what kind of connection should be used for accessing the database. Then we have data type mappings, which are used again for typecasting and other such things. So let's take a, let's start and take a look at this engine skeleton. So this is not live coding, but this turns out to be hard sometimes. So this is a basic engine that we have over here. Right now it doesn't have anything, but in our, in our module Tom, we have this engine over here. It'll have more methods as we go ahead. Next we have our model over here. We are defining bare bones, crud operations over here. And thing to note over here is the Arial table that we have defined, the table we have defined over here. So if you, if you see over here what we're using is we are making use of Arial table and we're passing the name of that particular, name of the table which you want to access from the database. And next thing that we're passing is Tom engine, which is going to, right now it is empty, but later on we are going to fill it and it is going to, it is going to be used by Arial to find out information about coding on that particular column and other such things. Here we are just passing, right now we are just passing appending an S to that name of that particular class. So if it is post, it is going to be post active record. You make use of active support for inflections and finding out properly how you're going to pluralize a particular table name. So yeah, that's the basic thing we are done with the basic engine skeleton setup. Next thing that we need to define is database connection, how you're going to connect with the database. So yeah, we have gone ahead and defined a Postgres adapter we are going to work with, we are going to work with Postgres over here. We are making use of the PGM as you can see over here, we have hard coded database name which is Tom development in our case. In, in the case of active record you're providing, you're using database.byml and it is passing and then from that it is determining what, what, to which table we are going to connect. Another thing to note over here, we are defining visitors. As I told, like, as I mentioned earlier, visitors are used by ARL for determining what kind of, for, in case of Postgres, what kind of, what kind of syntax should I be using? So here we are defining that our visitor to be used is Postgres. So it will be defining, it will be passing the tree and it is going to define statements in terms of Postgres. We have some other, other simple methods over here for quoting of table name, column and quoting of values. Right now these are very, very basic and we are just, we are just going to say whatever value is coming, just return that value. Here we are quoting the value, we are just going to mention quotes for, this is basically when you are creating a query for insertion, the value should be coded. We have gone ahead and started using this Postgres adapter which you are seeing over here and defined a method called as connection. You can see over here we can, this now is going to be exposed as an adapter and it is going to be used by our ARL table which we had previously, previously being passing into the model. After we have set up the database connection and it has information about accessing the Postgres, Postgres database by use of PGM, we are going to define columns. So let's take a look at, oh, so we are going to define this class over here which has information, which has three basic attributes which is name, SQL type and CAST type. Name is the actual name of the column, SQL type is in case of, in case like the actual database type, it can be var char, float, decimals, so on and so forth. CAST type is, we are not passing it right now, we will be passing it later. CAST type is, how we are going to, we are going to define types which are going to take care of type casting to and from database or from a user to the database. After we have defined this, we are going to define something in our Postgres adapter. This is a big carry, you don't need to know about it, you don't need to worry about it. What we are doing is, this query is being extracted from right out of ActiveRecord. What it is doing is, we are passing over here the name of the table, we want the information of that particular table and it is going to return information like the name of that particular, name of that particular, information about the columns, like the name of the column, the data type of that column, other information like default data type, like default value or what the value should be in case of a database. Right now, we are going to, we are not going to worry about all these other things. What we are worried about is the name and we just want the data type which here it is showing it is integer. We are using this column definitions and we are iterating over all the, you can see over here, we have array, we are iterating and creating, we are creating a particular, we are iterating through all of these and we are defining columns for all these different values which are being written to us by column definitions. So after we have information about these columns, which is an array of, as we have seen in the column class, we are going to go ahead and define a schema cache. So every single time you want to access information about a particular column, you don't want to go ahead and query the database and get information about that. That's why we are going to define schema cache, which is going to act as a, it is acting as a simple hash which we are, which is going to define, which is going to associate all the different columns that information that it has along with the, along with the table name over here. As simple as that. So the next thing that we have is after we have set up this basic things, which is schema cache, you can see over here, what all things we have defined. We have defined the postgres adapter as you've seen over here, which has basic information. We have defined column, we have defined engine, we have defined a model. We are going to go ahead and define some basic tests to cover basic scenarios. So to start with what we are doing is we are defining this model over here, which we'll be using in our tests. So we are defining class post and inheriting it from top model. So this is a basic model, which we are going to use in our tests. We are defined our test over here. You can see post model over here and we have defined some basic tests, as you can see over here, where clauses, which we have not defined yet, setting of the error table, which we have seen, initializing like what we essentially want to achieve is something like when we are doing post.new and passing this value, we are able to access this value on that particular instance that we have created. Right? Saving of new records, deleting of records, which we'll be taking a look at in the next slides. So till now is everyone with me? Yes, no? Okay, great. So this is a basic structure that we have right now. Now we are going to build on top of it things like attributes. So please bear with me. It's like it's like a lot. Sometimes people miss it. So after we have this basic structure setup, what we want to do is we want to define attributes. Attributes are the basic unit of access of a particular value on an instance. So when I do post.new and pass some arbitrary values to it, author, subject or something like that, and from that particular instance, I want to access the value like post.name. For doing that, we want to define attributes. It needs to work both ways, that is from the database and from the user provided values. For that, what we are going to do is we want to support data types. We are going to handle only two data types. One is string and one is integer. Another one is integer. Casting to and from the data types, database types. Attribute maps, so we have lots of attributes providing a simple access for all the attributes. Accesses for that particular, on that particular model. So as I said, post.name, that is an accessor, that is a method for assigning or reading a value from that particular instance. How we are going to achieve is we are going to define types, type map, attribute, attribute set, which we will be taking a look at now. So let's go in code. Okay. So to begin with, we are going to define a generic type. You can see over here, which has information about type, what kind of type it is, type casting, for a database. It has basic three methods, which is name of the type, type casting for a database, type casting from a database. This is generic if you don't know information of how to handle a particular type. We are going to just use this type casting, which is taking in the value and just returning that value back. We are going to define two other types in our case, which is, first of them is integer. You can see we have defined type being integer over here and type casting for that particular value, simple type casting, which is taking the value and determining how to handle that value. You can see in case of Boolean, we are doing one on zero and in other case, we are just using two i. Similar case, we are going to provide type casting for strings. We are defining type cast name as strings and we have simple mechanism over here based on different scenarios, how to convert that particular object into a string. After we have defined these basic types, we are going to define a type map to hold together all of these types that we have defined. Again, it is just a basic hash for which allowing us to access the types easily. Two important methods to take a look at over here, it is register type, registering a new type. In our case, we will be registering two types, which is string and integer and another one being lookup. Here, what we are doing is, what we will be doing is we will be passing name of that particular SQL type and on that basis, on the basis of the SQL type, they are going to determine what kind of type casting object, like is it a string or is it an integer and on the basis of that, we will be returning that particular class for use for type casting. If nothing is matched or nothing is found, we are going to return the default value, which you can see over here is just the value, the generic value, which is going to take the value and just return that value. We have a generic type set, integer and string types. Next, we have the type map. After we have the type map, we are going to go ahead and define our attribute. Attribute will be consisting of three particular values, name, value before type cast, without any type casting mechanism and type. Type is again the three classes that we have seen, string, integer or generic. Here, we have all convenience methods, which are initializing over here, conversion from a database or from a user. Again, these are all convenience methods, no need to worry about them right now. This is the basic unit of, as I spoke earlier, for handling of a particular value. All of these attributes, we are going to club together and provide something called as an attribute set. When we are creating post.no, we are providing a hash. All of this hash is going to convert into individual attributes and they are going to club together into what is called as an attribute set. Again, it is just a hash and it has information about how to write and read from a particular hash. Important thing to note over here is we are defining attributes variable over here. In case of active record, if you have seen, if you do post.no and pass some values and on that instance, if you do dot attributes, the hash which is being returned of name and value of that particular whatever is being stored in that instance, this is something similar, this is something similar which we are trying to achieve over here. That's attributes. So, after we have attributes set defined over here, again, we are defining some convenience methods for taking in values, hash of values like name and what value to assign and using it for building our object. Here you can see we are taking, when we take value from the user, we are going to build attributes from that values, which is we are just going to iterate through all those values and build one single attribute for that particular value. If you don't have value provided by the user, if we have five columns and user is providing only two values, we are going to initialize the other values with nil, like nil values, like the name of that particular type and initialize with nil. So, after we have this builder set up, right now we are on the stage where we have attributes, attribute sets and we are able to provide attributes to a particular instance. Next thing what we need to do is provide a database with readers and writers. So, we are going to go ahead and define something called as read methods. You can see over here which are singleton methods for if you have ID, name, author, they are going to define, just going to define singleton methods on that particular instance for accessing those values. Here you can see it is just fetching the value from the attributes which we have defined. And then we have write methods over here, again, which are singleton methods if you can see over here, we are just taking the values from the user and assigning them, creating a new attribute. You can see over here based on whether it is from database or whether it is from user. We are going to convert and type because that is basically being handled over here. All right. So, after we have these readers and writers methods set up, when we do post.new and passing that values, we need to handle how the initialization works in case of TomModel. So, we are going to go in our model now. Yeah. So, here you can see we are passing, if you pass hash of attributes, what it is doing is, first of all, it is going to initialize with nilattributes for all the columns that it has. It is going to initialize attribute methods, readers and writers for all the columns. Then it is going to take all the values that are being passed by the user and or in case of database, database and initializing like loop through all the values. You can see away the stringifying all the keys just for convenience and it is going to assign all these values that is using our, we have previously defined our assignment writer methods. We are going to use the writer methods and assign these attributes by just doing a public send. So, hopefully now we have reached to a stage where if we do post.new and pass a hash, we are able to, after we are doing, we have done that, we are able to access the value which is being passed to that particular new object. So, if we go back to our tests, something like this, like post.new, subject, something, something, this test should pass in our case because now we have provided the reader writers and handling of the attributes. So, let's go to our terminal and take a look at the tests. Yeah. So, we are able to now handle the situation where passing of attributes is being converted into a model. The one test that you're seeing failing over here is creating of a new, like saving, persisting it to database which you have not handled yet. So, all the other tests are passing. So, next what we need to do is we have this whole model thing working. Now we are going to work on persisting that is saving to the database and back from the database. So, things that we'll be handling is CRUD operations. Yeah, let's go back to the code. All right. So, we are defining, we have defined some basic operations as you can see over here, CRUD operations, create, update, destroy. Let's start with the easiest one, which is create. Again, create is going to take, if you know an active record, it takes a hash of values and it is going to create a new object and return that object. This is something what we are going to do over here. We have already seen, we have already defined new. So, we are going to, first of all, in case, we are going to take the values and create a new record. We are going to make use of ALS insert manager and going to, so you can see over here, we are using insert manager over here and going to provide information to this insert manager on which table we want to insert this particular query, what values you want to insert. So, from that record, we are going to extract all the current values that it has. You can see over here, it is using current attributes. It is just going to loop through all the values it currently has and pass it as a hash, which we are passing it to the insert manager. The insert manager on firing two SQL, it is going to return an SQL query and that is something that we are going to execute. After the create statement, we are going to return, we are not going to worry about what is being inserted. We are just going to return the last record which is inserted. So, we have defined last method over here. Last method is making use of primary key sequence. We have defined a custom sequence over here, which is going to increment whenever a new value is being inserted into a table. We don't need to handle this in case, like, if you are defining an auto increment on a primary key or things like that. So, yeah, we are making use of primary key sequence over here and we are passing it to lastinsert.id, which is going to execute this query and return the last value for this particular ID. So, this last ID that we have, we are going to pass it to find. Find is select, basically select. What it is doing is, we are using it where, where over here, which is being provided to us by Arial. It is just going to say, like, convert your tool and say where ID equals to so and so value. It is going to build us a query over here and we are going to say project. Project is another term from Arial and we are seeing that project all the values, like, give me all the values which are present in that particular, in that query, I want to find all the values. This find clause, again, we are going to go ahead and execute query that is passing this, passing it to the connection, like, pggem, which we have defined. It is going to, we are using two SQL over here, which is converting that into an SQL query. It is going to fetch all the values, which is, again, a hash. From this hash, again, we are passing it to new and it will be creating a new object for us. So, here, we are being returned after insertion of a new object. We are being returned with a new object which has information of last inserted object. In similar way, we have update. You can see where it is taking values. It is using update manager from Irel. And, again, we need to provide table information, what ID, for which ID we want to insert that update for that particular value and what values are needed to be updated. Yep, in case of, in case of this, we are just going to say find and the ID of that particular value. Similar way, we have destroy, which is taking in, which is going to make use of delete manager and it is going to say from which table we want to delete which value, like, in case of, here we are passing ID, so it is going to say where this ID delete this value. Yep, that's about it. Like, we have create, update, destroy. Next thing that we want to see is save. Again, it is a convenience way when you do post.new and you have the instance, you say save. If it's a new object, like, if it's already, if it's an old object, if it is being persisted to the database, in our case, we are just going to handle persistence by identifying if the ID value is there or no, in case of active record, it has a lot of checking, like dirty, if the field is dirty or no and all those things. Right now we are just going to say if the instance that is currently there, if it has information about ID, so it's being there in the database and we are just going to say update, which is this method over here which you have seen already. Else we are going to say create and go and create a new object and return it to the user. Yep, so we have seen these basic code operations. To support us in our tests, we have different operations for querying, find count, where delete all, these are some of these operations where Arial provides us with a method for where, which simplifies, which is just going to rotate through all the values and going to say for where clause, this value equals to this value. Here we are just, we are going to take a hash of values and convert it in, so that it is compatible for Arial. Count, we are just taking the name of that particular table and executing a simple count. Delete, we are using delete manager for deleting all the values. We have seen already, we have already seen primary key sequence. So let, so if we go back to our tests, we have different scenarios over here, which we have seen earlier, like initializing to an empty object, creating of an object, like you see over here, we are using delete also with, it deletes all the values and then we are going to create a new value and it should be persisted. Yeah, updating of a value, deleting, destroying of a value. So if we go back and try to run these tests, all right. So it is executing all these statements, which you can see over here and we have a basic tiny ORM, which is able to perform all these credit operations. And if you'll take a look, we have achieved this in only five, 60 lines of code. So that's about it. We have taken a look at database connections, creating columns, schema, cash, attribute types, attributes and attribute accesses, database statements and querying. There are other different things like associations, migrations, callbacks and validations, which we are not supporting right now, which can be, which are being supported by active record, but which are trivial to handle. And that's about it. Thanks a lot.