 Thank you So I'm Emma or I'm a new L if you really wants to have a mouthful I am the co-founder of a small company in Belgium You might have heard of me regarding one of those two libraries the R schema adapter in embers. He liked read it He's I am not going to talk about that today. So if you want to talk about those libraries to me just find me in the holes and Today we're going to talk about factoring and as I said we're part of a small business and We do mostly remote work and sometimes we Find projects that have already been started for a while by different teams and sometimes it's a mess And so that's how I got the idea for this talk once I Got into a project I came I came there and say oh, yeah, we have to improve the maintenance and availability and performance of This feature and I looked at the feature and it was this method that was several screens long with nested ifs and fordubs and stuff and it was not readable and I had no idea where to start to get it faster because I could not read the code So before we get started what is actually refactoring? Well refactoring can be many things If somebody tells you hey, this is the one recipe for refactoring Well, don't listen to them because refactoring is a lot of different things and so there cannot be only one recipe Refactory can be something refactoring can be something as simple as renaming One single variable all over your code because you decided that finally that name that you gave it was not that smart of a name after all It can be updating your cook style and making your code base that they've compliant for example if it was not for starters That's that's one sort of refactoring too Refactoring can also be writing a single method function of class to make it more maintainable Just one you don't have to do the whole code but refactoring can also be rewriting that same thing to make it just more readable and To make it easier for other people to add other features and that's mainly what we're going to be looking at today And finally refactoring can be rewriting almost everything just keeping the name of the project And so this is the kind of method that I want Yeah, it's sideways. Did you notice because I didn't have enough room to put it vertically so I had to put it sideways this this is the kind of method I was talking about and Other than the fact that it's sideways. You probably are having troubles reading all of this You have no idea what it does. It's just comments. It doesn't do anything And This is something that is really common actually to thank this and it is so common that people on the internet has Made challenge some kind of challenges It's not really a challenge because there is no winner or loser But it's an exercise to make you refactor some codes. This is this one is a gilded Roscetta There is going to be other ones. I Forgot on the first slide there was the The URL for the slides I will go back to the first slide at the end of the talk All these are working links so you can click on the slides and you will have all the links, but you can also take pictures So this is the first link is where I got the code It's really nice repo with the original code in many different languages including Python 2 that tree sorry But we will work with Python 2 and the second URL is Where I pushed all these steps of the refactoring that we are going to do together But this is the first section of the afternoon of the last day So I guess I need to keep you awake. So I'm going to be asking you some questions I'm going to ask you to participate. Are you ready to help me refactor this? That's for people Okay, thank you So this is a new formula that I'm trying today I hope it works well First of all, so we're going to read the requirements of this Refactoring because we're new we were new we're new in the company. It's it's looked fine We we got a nice contract, but it's our first day on the team So here's the assignment and I'm going to read it with you Hi and welcome to the team gilded throws As you know, we're a small in with a prime location in a prominent city run by a friendly in keeper named Allison or you can Don't call me Allison. I'm a by it's not We also buy and sell only the finest goods and thrown it unfortunately our goods are constantly degrading quality as they approach their sell by date We have a system in place that updates our inventory for us. That's good It was developed by a non-onsence type named leave right Who was moved to was moved on to new adventures? Your task is to add a new feature to our system so that we can begin selling a new category of item first an introduction to our system and so this document is very important because that's Basically the requirements for what we are going to have to do for the next hour And we're going to go back to this document every so often The first thing is that all items ever selling the value which denotes the number of days we have to sell the items by Then the quality degrades They have a great evaluation which denotes how valuable the item is and all items degrades as the sell by dates Comes closer. I'm going to stop reading now We will come back to this later, and we're going to start looking at the code for this What do you think Do you want to touch this? No Yes, that's okay. That's that's a good that's a good start We're going to do test we're definitely going to do test and what is nice with this is that? Somebody actually did us a favor and also provided some fixtures so we can run The current code on the fixtures and have the output store the output because right now there is absolutely no test on this code So our baseline is that we've got some fixtures, and we know that the code works So if we change something we have to be able to replicate to the same results But before we did that This is some kind of simple example, but something that I like to do is that I like to look at the complexity of the current codes and There's a tool for that in Python. That's called Fredon. It's Linked in the in the slide and Fredon allows you to calculate the complexity of your code in various different ways and this is Computing the cyclic the complexity And you see that our update quality methods got a C. So it's not very good. I try to often run my entire code by that and in Prediction code is C is okay, but when you're starting to get D E and F. It's really bad But as One see every once in a while can happen. There are other metrics. This is the one we're going to be using today and we're also running the fixtures so You can run the fixtures for as many days as you want So here the first thing I'm doing is that I'm running all the fixtures and storing the results in a text file So that we can just update our codes We run the fixtures and do a VIM diff between the two text files This is not tests, but when there is no test Being able to output the result of something and compare it later and doing a VIM diff between two things It's a hack, but it works and it can save your life No next thing I want to do we said we wanted to to write some tests and To be able to write some tests I'm looking here at the test fixture program that was Sent to us I'm not able to resize that window So we've got all the items directly in the test program So the first thing that we're going to do is that we're gonna take the data and put it somewhere else so I'm taking all this and This is live Which is probably a bad idea. I didn't sacrifice any kitten But this is only going to be the only live part almost of this talk So from killed it rose Import item since this is the class that we're using for all our data is a class item and Back in this Here we have to import items from Data and as you might see my my VIM is not I have two spaces. It's okay My VIM doesn't really like the code It's not pepe compliant that's okay. We don't care for now So I'm going to quit all this to save it and quit and see if it still works. I Guess I have to quit both files So let's run the test again and It doesn't work because I broke my code. It was a really bad idea to do that part live So Let's go back to Syntax error import items from data from Yes, that's thank you. Thanks a lot. That's that's that's why it's fun to do things like that It's not the right file And the fun thing is that I don't know why but I don't see the last line of text on my screen right now And who could guess what language I've been programming in for the night for the past few days Drivers, thank you. And so if I run this Syntax error were expected Okay, let's let's just Trust me if we got the same results because now I'm going to check out the next I have two bitons Yes, let's Let's not worry because if we do we're going to run overtime. I'm going to Just check out the next revision because I did plan this I did plan that my coding was going to be awful when I was live So everything is in a repo what I'm going to do now is basically I'm going to go and check out the next revision each time So Here we have our data and so we would like to write some tests The first thing that we want to do since If we go back to our description we are going to be able to see that Once the cell dates as past the item decrease twice as fast for certain items The rate is different and so on and so on so the first thing I want to do is that I want to write a few test Helpers that will allow me to get the different items that I need to be able to test their behaviors So Back to my test here, you can see that I have already written some code this one is gonna work and Here I'm going to stop and say If you want it a generally factoring talk You're in the wrong place because we're going to do a lot of things that are only just working for Python Because Python is going to be really helpful. I said this is An exercise that's available in different programming languages and Python is really really helpful to give us things that will allow us to work faster for example We have the first thing here We are declaring a property which is not possible in all languages and we're declaring a property on the test And this is going to be our regular items that age in the normal way and It's a simple list inclusion I'm going to Switch this so what I'm doing is that I'm Just returning all the items that are not special items This is a simple list inclusion It's everybody here familiar with this inclusion or do you want to know how they work? Okay, and so the test helpers We're going to do that for every type of items that we want to check And what we want to check is that the quality increases or decreases as we are told it should do so we're also writing a few other helper functions That are going to to compare the original quality or the original setting date and Give us the list back This is some helpers and Then once we have that we can start writing our actual tests and Maybe something that was not sure that all we have all the items on the side and as you can see the only Way that we can identify which item is what is by their name So it's rather simple. We just have to pass the first word of the name of the item So it's everybody Freddy to go on Yes, three people I lost one Okay So the next thing is that we're going to start actually writing tests and we're going to going to test that the quality of regular items Actually decreases so As you might have noticed And as some people have said in the audience Testing is going to be the main thing that we're going to do before able to refactor So that's about half of the code that we have to write and I'm not going to explain all the testing details If you want me to explain I'm going to show them all but if you want me to stop and explain one in more details Please raise your hand or shout or do something because I cannot spend all my time explaining the test Or this is going to be very very very very boring for everyone So we've got our first test and it passes. Do we still have the video on the side? Okay, great So we've got our first test and it passes this this is the road to progress Next thing to do is to test that Quality degrades, but also the sell by the degrades. It's almost exactly the same test So while we're doing refactoring, let's refactor our tests and write a method that takes an attribute which will be what we want to measure and make sure that it decreases and then call that so let's run that and It fails oh That's that's not fun. Why why does it fail? What what does our cheat sheet say why why would that fail? Oh? Here we this is the reason why See I remember I looked at it before I knew where to search So the HV item actually increases in quality the older it gets so That's one. There's also This one the sulfur is I didn't know what the sulfur is never ages It doesn't change it doesn't decrease in quality. It never has to be sold So this is this is our culprit. The other one is about quality, but this this one this one Is the one that is posing problem so Let's fix that And so we need to to declare another helper. That's non-silver's item. So that's the item that do age correctly and Instead of testing on all items are just going to test on Test on this one on the non-silver's items So let's go back to our test. We now have to test passing This is progress. I still got two people. I'm going I have I have to to increase the speed of Talking about the test So we're going to be testing other stuff like the regular items decreasing quality twice as fast when they are explored and We've got three tests passing next one When does it stop there? We're going to test that the quality is never negative because Quality is something that doesn't go below zero. It either doesn't it's not worth anything or it's worth something But it's not worth minus something French cheese who likes French cheese Okay quite quite a lot of people about half of the people so French cheese you might know It increases in quality. It's like wine when it gets older So let's let's have a test for that too And then we have all the other specific things Like the one we mentioned the sulfur is that never changes The back six stage passes that are quite complicated This is the most complicated test we have the back says the backstage passes The increasing price in quality the closer you that you get to the date of the concert But after the concert, they are not worth anything It all makes sense So we've written all the tests for everything and we now have a test passing This means that we can finally get started Well what time is it has been 20 minutes So we spent one third of our time just testing what we had no we can we can start actually doing some work And the first thing we're going to do is to write a failing test And so to write a failing test we know what we have to know what what we need to do And so we have to to add a new feature this one We have to be able to sell conjured items which degrading quality twice as fast as the non normal items So that sounds easy, but none of you wants to touch the current codes. I don't either So let's first Go back to here and write our test for conjured items So I guess there are two different cases that we need to to account for We first have the non expired items that needs to decrease by two since they age twice as fast and The expired one have to decrease by four I'm also adding some test data so that we have some more complete test data and We can test that and we can now see that we have our failing test. Yay Thank you. I've never seen so many people upload for a failing test. Thank you Next thing and this is the second thing you can see the the threaded message there on the bottom right live So this this may fail I'm going to actually write the code for that If you if you notice something we are dealing with items and We already have an item class at the bottom of this file What do you think we should do how could we Endle different things that are almost similar but act slightly differently subclasses yes, and so since we have subclasses means that We're going to have to shift the responsibility of aging from the main Collection to each item. So that's what we're going to do right now And I could very much delete all this code I Should at some point, but what I'm going to do for now is that I'm going to comment it out just in case There's something that goes awfully wrong, and I have to take a look at it I really don't want to do that, but just in case I'm going to to keep it and comment it out and So since we know that we're going to do subclasses What I want to do is I want you to just start with item and define What does a regular item do so? Let's see your if you're still awake. What happens to the quality of a regular item every day Decreases by one so dev update quality self self dot quality Minus equal one what happens to the selling date. Yeah, it decreases by one Selling yes selling sell by its So self dot sell in Minus equals one Okay, so that's for regular items and we're going to start by with that and it should pull is still a few of our tests should probably still pass but Most likely not all of them So I see what happens we have six failures Six failures out of nine. That's one third of our test passing. It's still still good I'm sorry. I cannot hear you Okay So the next thing that is that I remember for the definition is that Is this one and we're going to take them one by one once the sell by date has passed 20 degrees twice as fast We could do a new statement in the update quality, but that's in the end that would bring us back to where we started So this is this is not really what we want Does anybody has an idea how we could implement that? Yes, exactly. So I'm going to say out loud. So what was proposed is a degrade method or property that would calculate by all much each item should be great and We can do that and Except that we're going to call that freight to know at which rates it degrades So this this here we have a property rate now And if the ceiling date is less than zero then the rate is two otherwise the rate is one and Every time that we update the quality we update the quality by one times the rate That's good. It's easy so far to do refactoring so Five failures we fixed something great Next one The quality of an item is never negative Well, we can do something complex with a main or something, but we're just in this case We're going to add an F so a Simple test at the end of the update quality method if the quality is under zero. We just put it back to zero and for failures Really easy so far does it get more complicated afterwards Oops At the right screen. The next thing we have to take care of is That French that French cheese again, which actually increases in quality Can anybody have an idea on how to handle that? Subclass. Yes. Thank you. What people is still awake. Can you make some noise to tell me you're still awake? Okay, so subclass aged item Here underneath it and the decrement for that we're going to add a decrement property to regular items and We're going to run the update quality with that property and for aged item is going to be minus one instead of one So if we decrement by minus one we increment by one we make the quality go up So this should work shouldn't it? That doesn't work Why No, no, we had four falling tests with six failing tests It's probably because I I I didn't show you all the code We we know we have items. We need to get the proper class for those items so We are With I've added a method and I'm going to show it to you It's really hard to type something when you don't see so there's this get specialist item for which is a helper method Which will help us get the the the regular item the item for the class So now this means that we're making a copy of the original item and we are updating the copy of the original item And all our tests are done on the original item. So what we have to do To fix our tests Is just make sure that you are testing On on on the copied items so here The code previously was for item in items And items was fed to the original class And now we are going to test for item in gr.items So which means that we are going to actually do the test on the item that's updated This is something that is easily That's very easy to get from when you use a refactoring You say, oh, yeah, I'm going to do something different I'm going to do subclass and then you copy something and You wonder why nothing is working anymore. It's because you're updating the copy. You're not updating the original So you have to make sure that you test the right thing So we're back to four failures Okay, we're back to four failures That's the problem because we had four failures and we were trying to fix something What else is wrong? Well, I could ask you But I'm going to show you It's easier. Here we have The test for quality of Of aged items That was failing and now we still have four failures But the third test that fails is That the quality is never over 50. So we actually solved the problem, but we created another one So We're going to fix this quite easily We're going to do with like with quality It's never less than zero quality can never be over 50 if it is we put it back to 50 Why did I use self the max quality in some places and 50 in other places? Because I didn't do that on purpose But so does that solve anything? Yes, it solves it. We are now down to three failures We're making huge progress So the next thing is to write Why am I showing you the answer already? You have to participate. I'm I'm hiding the answer Okay, so the next thing that we're going to do is we're going to read the next line Sulfurers does anybody know what the sulfur is What is it? It's a helmet Okay, thank you. I didn't so there are there are legendary items and They have never have to be solved. How can we solve that? you've probably seen it it's The update quality basically during the update quality method. We have to do absolutely nothing What do we do? Yes, subtype and overwrite So that's what we're going to do. We that's probably the easiest thing to do We subtype and we create an update quality method that does nothing Okay, so that should bring us to Two failing tests Yes two failing tests So what was the last one is probably the more complicated one is the backstage passes Backstage passes are complicated stuff So this is probably going to be the The most complicated part I'm going to go relatively fast and tell you the answer. We are going to subclass and overwrite the method But we're not going to overwrite everything We have two different things to do one we have to increase the rate as time passes and then We have something else to do if the cell by date is over the quality then drops to zero So The way I chose to do that Is to have if statements in the rates And just check the selling dates in the update quality method and call the parents if The cell by is not passed You probably this is A lot of you wanted to do the same thing as me. This is not the only way you can refactor that This is the way I chose to This is the way the gentleman over there chose to do it apparently too because he's been Saying a lot of things, right? But you can do that in other ways. You could have Completely rewritten the update quality method and everything but This is this is how I did it So, uh, no, do you remember at the beginning we said that we had Okay, I don't know. We only have one failure where we're back at the beginning We we now can start working Um, and do you remember that we said that we had text fixtures? So now it's a good time to check the fixture output And see if we have the same thing We don't have the same thing But if you look at the right part of the screen, I told you that I did add some fixtures So the only difference is between the two files or the fixtures The the three additional fixtures that I added So this means that this works Great. No, I'm going to ask you again Now that we've done a lot of the work How can we implement the last feature? Am I right? By changing the rate. Thank you So we're going to try to do that Okay, and this is where I messed up my script I didn't want to solve the This problem yet because one thing I didn't show you and I didn't mention Is that I got a little note here at the bottom Feel free to make any change to the update quality methods Right as long as everything still works correctly or if I do not alter the item class Or there is a gobbling that's not going to be happy. We did change the item class Oops So did did we write all these codes for nothing? Or can we try to fix this in an easy way? Yes, we try to fix it. But how? Okay, so here in python, there's something that's uh, there's a design pattern that's quite easy to implement It's called the proxy pattern So everything that we wrote for the item class, we can just move it down to an item proxy class And the item proxy what it does is that it takes the an item As an input In the init method it takes the item And right when you try to get the quality or the selling date on that item proxy Is going to give you back the quality of its item And when you try to set the quality or the selling uh date on that Proxy is going to set it on the item So what we're doing is that we're wrapping each item in a in a proxy. It's like If you wanted to put it in a big plastic bag And when you look through the plastic bag, you see what's inside of it But it's just inside the plastic bag. So we don't change the item Class we just wrap it in a proxy And everything that we wrote All the specialist class they can be specialized proxy and we can wrap any item in its own proxy That way we don't change anything to the item class And the only thing we have to the only other thing that we have to change to make this work Is that we have to change this get specialized item method and Let it return an item proxy instead of returning a Returning an item um, I guess I didn't explain this so, um There's a regular expression who needs uh, who is familiar with regular expression Okay, most of you for those of you who are not This basically means that we're taking the first word of the name of the item That's that's what this regular expression means then We try to Find a class that's we're going to be using and we can not use class with a serious variable name because Yeah, it's already used for something else um And the class is going to be item proxy if There is no first Word in the name. There are some special cases that start with plus five and stuff like that So there is no first word in the name or otherwise we're going to look At all the variables everything that is declared in the current module And try to get an item proxy by that name like h item proxy or something like that And if we cannot find it the default value will be item proxy So everything special you give me the special proxy everything non-special you give me the regular proxy and so we can Re-update our tests So that's um the test friend on The test friend on regular items as well as the the fixtures we can Run them on items and we don't have to run them on gr.items So that that that fixed our issue with the goblins because who wants to have troubles with goblins Okay, so now let's actually fix this and let's go back to the solution which is So the fixture back there Which is to add Different rates for our conjured items And so all we have to do is to create a new class with a different rate So the new feature actually implementing the new feature is two lines of code Only two lines of code All the everything that we've been doing for 45 minutes has been Just getting getting us ready to do that So why did we spend 45 minutes doing all that? But because we're going to keep working for that in and we did nobody here wanting to touch that code No, we did spend 45 minutes which we Would probably have spent to read what the method was doing and try to understand it But now we have some clean code and know that we have to make a change. It's only two lines And just to make sure everything passes the nine test pass And finally, I I'm a bit curious You remember at the beginning I ran the complexity test So we get something that are a bit complex. We are getting a specialized class. We have a method for that Um, what is the complex the complexity of the new code? Hey, hey everywhere. So Oh, thank you um So of course in real in real life code a everywhere is something that you cannot achieve You will probably not achieve that It should not be A metric in itself to try to well to have all a's but try to have as Few d's e and f's as you can And stay everything mostly in a and b and sometimes c There are other things. There are other uh indexes that you can measure with radon. I'm going to Show you uh, so here we have been uh Measuring the complexity. It's pretty simple every time you have a need for a narrative It's you add one point and so that's how the complexity is measured and then it Makes some computation with the number of lines and everything this library also Gives you a maintainability index. I'm not going to explain that formula. Um, that's um Something that's not useful for this kind of refactoring But it's basically based on the number of lines in the file and the number of lines in methods So it's going to be really helpful to have that the tool is called radon or adon You will have the link in the In the slides and just to make sure We can take a look at The side by side we have a lot a lot more items On the on the new code, but everything every new item is very simple. So now you see side by side and uh That's about it. So I was a bit fast So we have time for questions If you have questions, please lean up Thank you. That was an example where you have like a really clear specification Yes, but what strategies do you use in order to get the specification out of the initial mess Okay. That's that's a very good question. Um, it the answer is it depends If you have something that you trust in the organization that you can go ask questions You can go try to try to find this information If all you have is a bunch of documents Uh, you're better off, uh, trying to read the the actual code the current code trying to understand it and the part that you don't understand Don't spend too much time trying to understand it. We in this case, we had some fixtures You probably have in whatever context this is you probably have some data and you can just run The existing code on the existing data see how it behaves and get Try to guess the specification from that Depending on your organization There's some somebody who should know these things and you can ask maybe it's not somebody you trust but somebody from the business side Uh, if you're in a in a large company, you're in the it department You're told to do that you don't have all the specifications because someone left Well, this is uh for the sales department Get up go to the sales department find the person who's using this thing and ask them How is this supposed to work? um There is no miracle recipe it really depends on Where you work in which context But there is always at least somebody that has a big idea of how it works either. It's the final user Either it's your client that you might not be supposed to talk directly. You are supposed to talk to your manager Don't call them send them an email or something But try to get in touch with the person who actually needs to use that code Hello, um, I have a question related to the business side of the justifications So obviously as an engineer, we wanted to always have the clean code And then if you go to the the client and look at the code Oh Your whole code need to be rewritten and reflected So how do we handle that pressure like business pressure say you pay me the money? Suppose I just add one picture however you in the end you said Like this every code need to be Changed then and uh, they they say Hi, you just for I didn't want to line but use in the end you change everything and take that long time and then It's it once again It depends in the the example that I gave at the beginning of the talk the client Are it me to make things more maintainable and faster? So Rewriting is part of my job Um, when it is not explicitly part of your job It's hard to justify that you spend Three three fourths of your time refactoring something Instead of actually working on the new feature. You don't have to rewrite the whole code of the client Here this was the whole code of the client we reward almost everything But most of the time your feature the feature that you're going to have to implement Is probably going to be one small portion Is going to to involve one small portion of the code of the client So, uh, the best way to go about it and it it also it once again, it also depends on the client And oh, uh, they understand the need to have clean code and stuff if they don't understand Uh, just refactor as you need to so you are writing a new feature You really don't understand that method refactor the little part that you don't understand Create the feature and day by day the code will get cleaner Um, that's usually how I I do it when the client really doesn't want to hear anything most In my experience and not this is my experience and it is different everywhere, but Most clients most and And clients seem to be more and more Aware that they need to have maintainable code So it's easier now to tell the client that a I'm going to do a little bit of maintenance on your code Then it was maybe five years ago You're welcome Um Yes Hello regarding the radon tool is a static code and license tool I'm using Sonar cube for my project How does radon act with Django functions and models and so on? um Usually, uh You get a b Uh You get a b for most models It's hard to you to have models with a it depends if you have fat models or skinny models For everyone fat models is where you have all the functionalities of the model inside the models Skinny models is where you have a surface that takes a model and changes the model when you want to For example mark an invoice as paid So it depends but if you have fat model you can you can get away with b's and c's uh, don't expect an a And sometimes if this is something really complicated you might get one One d or something Thank you. You're welcome Anyone else? If not, let's thank our speaker one more time