 So, good afternoon, I think people are getting ready for lunch, so we are with two more sessions. I will try to make it fast so that I do not bore you with my talk. So, if you are back, we are working for a product and it is an e-commerce product. We are building a module for designing things online. So, the idea was you can design things using your browser and print it on your goodies. And not a, it is very common, but so the users can come to the site, design things and say that this is what I want, that you should be printed on the goodies and then they can pay it and then the design is delivered to the printing vendors and then they deliver the things get delivered to the customer. And they can do multiple things, they can upload their own images and resize it, add their own text, add different kinds of shapes and so forth. And it was working well, that is when we realized that we have a problem, that is we wanted to introduce the functionality to certain text transformation, it is not exactly transformation text features such as actually rotating the text or adding shadows to the text, stuff like that. And that is when we realized that the library that we are using for converting the design that you see on the UI to the PDF, that it was not very precise. It has to be very precise, the colors and the position has to be very precise so that the printers can print it very well, so that it does exactly look like how do you see on the site, it gets printed exactly like that on the, on the goodies. So, we hit with the bottleneck at certain transformations were not working as we expected and that is a huge change. And so we have two problems here. One is that the current library does not support the feature that we want to introduce and we want to release that feature as soon as possible. The next problem is that we already have a lot of features built using the existing library and then we also need to migrate that functionality to the new library. So, what we did was the immediate requirement or what business wants is to deliver the new functionality. So, what we did was we approached it using a concept called or a technique called parallel change. So, what it does is that we have both libraries sitting in the code base, the existing functionality uses the existing library and for the new features that we are adding where we had the issue we used the new library. So, this is the parallel change and over a period of time we migrated from the existing the old library to the new one and that give us enough time to test the new library rather than going to into a branch and doing the entire migration coming back and releasing and hit with surprises this give us enough time to evaluate the new library so that we do not hit with another another surprise down the line. Again this is this is not a very new technique this is called the standard branch by abstraction. So, do not branch the repository instead branch your code and over a period as you gain confidence you migrate it and then how long the parallel implementation should be there it depends on a lot of factors. And things moved on now I work for a product called Good Karma. It is a product that is delivered to yoga studios to it is a B2B product for yoga studios to manage their business and so we started still a start up we started 12 to 18 months back and so the product is evolving and we started with certain idea then we as we onboard new customers or we learn from the market we realize that okay there are changes and that means the changes needs to happen even to our database models. Okay so because of the as the product goes the understanding has to reflect on the product as well as on the modeling that you have done on the on the database. So and you know that database is considered even if you are a start up if you don't have millions of customers even then the slowest moving part of your product is database and that is considered as a very high risk thing because if something goes wrong it is very scary especially on the database side migrations and stuff like that it is very complicated and there is high tendency to become very nervous when database doesn't work as expected. So can we the thought process is can we actually make it less risky and that is the whole premise of premise of continuous delivery if there is something that pains do it all the time so that your pain gets reduced it's not the solution is not know to do that solution is actually do it all the time so that you don't have you you take the pain forward and do it continuously rather than doing it once in a while right and that is where the concept of database is refactoring comes into picture and the the key word here is refactoring the refactoring as per that Martin Fuller has explained the beauty of refactoring is that you do it in small steps you do a series of small steps refactoring which gives you a bigger result and that every refactoring that you do has to be taken to small task as small as possible and that is the value of refactoring if you do it in the actual way and similar things can be applied for databases to okay and that is what I learned during the last few months or say even even close to an year so there are techniques none of these techniques are mine there is a book called database refactoring it talks about how do you structure each migration depending upon what kind of migration you want to do right is it a and are you adding a new validation or are you changing the structure of the database or are you adding new index x how do you approach each of this is very well written in that book and I'll take up two examples that I've used recently while while building the product so this this is called the split column refactoring what it does is so the product has a concept of has a module for booking online booking of trial classes that is the customers can come to the come to they get an SMS to a certain with the link then using an IVA system they go to that site and then they can book a trial class with any yoga studio okay completely automated and what we need to know is that whether the customer has attended the class and and that is not where the funnel ends right funnel ends if they become a member of the yoga studio so first they book it get that they get the SMS that first step then they book a class then they attend the trial and then they become the trial that become a member and that's a success criteria of of of the entire thing right so we had a status field in our trial booking stable and the status fields stays whether they have attended or not along with the status saying that okay have they registered okay so it can be they attended but they are not ready to they are not interested in registering now or can be that I'm not I don't want to register now I want to register few months down the line then there has to be a fallout mechanism right so both these fields together we're stored in the both these data together was stored in this particular field which we realized over a period of time that okay we need to split it and so that we can run matter better analysis on that and then give more insights to our customers and that is when we had to split it and we called it as attendance status and membership status first attention stage says they whether they attended or not if they have not attended membership status may not make much sense but that is not mandatory even even without attending it they can still become a member and we so what we did was we introduced new fields still kept the old field and apart from that introducing the field we also added certain triggers or say even every ORM supports callbacks so that anytime when you save the data to status it has to split and store to these two fields okay so this is this is the first step that you migrate to the new implementation and then there are so many so many I asked as part of the code base or even other dependencies that you have they depend on this field so it takes time to actually change everything across right and even even how do you display on the field on the how do you display on the UI or see even other downstream dependencies that you have which uses the same field so it takes time right so it took you can leave it like that for some time and then until until all the downstream dependencies are moved to the new implementation and then you contract it to the new implementation so this is a end-to-end refactoring process you introduce a migration that is a transition period and at the end of it you contract it to the new implementation and so and rather than doing it as a first one step you are doing it in multiple steps okay so another example is split a turbo where in this case instead of splitting into multiple columns we are splitting into multiple tables so we had a billing cycles which had all the payment details we wanted to segregate it because later we realize that okay one people can pay in installments so one in when they are paying they can say okay I will pay now after a month I will pay the rest of them all so it has to be tracked in multiple tables one one billing cycle can have multiple payments so we had to segregate that the same technique we introduced triggers to save to the new table we changed across the application to use the new structure and then you move completely move to the new one okay so the same thing you have a migration time and then you completely switch back and that is where the the the entire process is called expand you expand to the new way and then you contract completely completely get rid of the old way of doing it and so the what is required for you to implement it what is the basics that you need so that you can always approach this one is a versioning you need to have versioning for all your database migrations every migration has to be uniquely identified and as as far as I know I might be wrong in this case but I think almost all the ORM frameworks supports especially those follow the active record model supports the migration way so that you can quickly roll back or say move forward or roll back to a certain state of a migration that's a basic need enough automated tests as safety nets to make sure that okay none of the none of the changes are breaking especially when you are transitioning and contracting okay this this gives you enough confidence and then the last is how long can can it how long it can take for a team to actually contract and that completely depends on various factor I've been a consultant so I I'll say it depends right so it depends on various factors and how long you want it to be how long the migration how long you need the parallel change to be implemented right it depends on how big is the team how much how much critical is the current migration how much time how much downstream dependencies you have how much time you you may need to keep it so that you gain enough confidence right lot lots and lots of factors that you need to think of on and this is not just restricted to the expand contract pattern it I think it is it is about how do you want to deploy database migrations right do you want to have down times then how do you plan your dime times or if you don't want to do a downtime how what is other mechanism of doing it and that's the strategy that you had to build and this is not the this this end up idea of parallel change or expand contract pattern is is is about this low lowing lowing the risks of every release right I mean that's the all idea how can you bring down the risk that every deployment every release is risky but that doesn't mean that we can't release right but you can only reduce that by doing it in the taking care of certain things and what are those principles one is doing it incremental and rather than doing it doing it in a completely taking it as steps itself is a first step and then making sure that any change that you're doing is actually ready to deploy that that making sure that every commit is has enough confidence that you can build that okay it goes to production it doesn't break anything so automated tests and other things helps you in this case and so you can actually you don't have to every deployment doesn't have to be resulting in releases right for example in the case of a migration we have we had actually migrated the database but the user interface or or or or or or it is not visible to the users that okay new changes have been built into the system right and then that is called the risk of dark launching when you're just deploying as a migration it is called the dark launching or even feature toggles help you in this case you can turn off that feature in certain environments because of various reason you want to you don't you're not ready for doing that and so decoupling having having enough mechanism to decouple every deploy from release helps you to lower risk lower the risk of your deployments and doing it in small batches as my as small as possible right and and that's so that's that's very very important without that I think the rest of the things will not work and as a reference I would highly recommend this has a lot of techniques on how do you approach database migrations and some more references and that's it we have time for questions so I can see the parallels between having feature flags and also doing this kind of step where you expand and then contract so what I see happening is over time you have a lot of code where we have things which are really not using you have expanded but not really contacted so how do you so the issue is like you have to remember to keep track of it so how do you do that as part of your planning process when do you how do you kind of do the issue tracking for those kind of things I think it was very important because we don't have do it right it's kind of becoming so that is the reason I emphasize on the strategy okay so what is the approach that you want to take what is the guideline that as a team we want to follow right the same same goes with it can be same goes with future toggles also you can say that no feature no feature toggles can exist in the code base beyond two weeks as if the entire team agrees that okay that's a good time frame then you may need to bring in system so that it is automatically detected or you may have to bring in some kind of system to know that okay there are certain toggles which are existing beyond this time so we had we had used our own backlog kind of thing for feature toggles and this kind of changes to make sure that okay you don't it's it's tracked separately but but interestingly I've heard that there are tools automated tools that certain teams built I've never tried that to check the code say for if it is for future toggles it checks how many toggles are there and then it if if if there are certain toggles which which beyond us expiry time then it phase built so you can go to that level so there will be some kind of a guideline that every team follows for either for expand contract or feature toggle and you can look for that patterns and then have automated systems to fill it that is very great but I've never tried that but having a backlog of yourself and then keep monitoring it when in the normal team meetings or the daily stand-ups bringing that every once in a while might after a while what I've seen is that it sets it down in the team right if you bring it very consistently then I think over a period of time it becomes a habit for that more questions okay thanks