 Hello to everyone and thanks for being here in my talk about 10 lessons that I learned during a big Django project, introduced myself, I'm Yamila Moreno, these are my Twitter and my githabagam, two years ago I founded with another Pythonist in Spain, the Spanish Python Association and we organized the first PyConness in Madrid and we also co-founded PyLay in Spain. I'm a software developer in Kaleidos, it's a company that I co-founded four years ago and we work only, mostly, 99% with open source and it's important and we'll see later. In Kaleidos we work with, we make web applications, big web applications usually for, with Django as well as other technologies, but Python and Django are part of our stack. So this talk is about the good and the bad decisions that we made during a big Django project. It's especially, the project was the complete renovation of a roadmap of a website in Spain, quite famous and it's meant for all of you who are starting with web development or maybe facing for the first time a big project. The website, this project is the Gear Repsol, in Spain it's quite famous, Repsol is a massive company and the Gear Repsol is part of the marketing strategy. It's a roadmap with, you can search for routes and it also has many advice of tourism, gastronomy and accommodation. So it's a baby. In this project we needed to change completely the brand and give a new style but also we needed to keep the original essence and the creative team focused on five main features which were tourist and gastronomic cards, search engine, a powerful map, a planner to organize the trips and looked after the content. I said it was big, how big? So we had 80,000 touristic cards, more than 500 articles and growing, this amount of data maybe is not so impressive with all the data but it was quite difficult to show all of them in a map at a time and very quickly with filters and everything. The first release took about 18 months and we spent another six months with the transfer to the new team and adding new features and in the present time the project is being evolved. The original team were two UX, one designer, one frontend, six backends, the product owner and as well as the core team we had the content managers, the translation team, the IT team from Repsol, the Academia de Gastronomía because we had lots of people giving their opinion to build the team, the project sorry. We work within the agile methodology so we use DSMoliteration which are sprints, two weeks sprints. We made 42 sprints, 600 user stories and more than 1,300 points. So this was the project, this is like what I mean with a big project and with this context in mind let's talk about the decisions. One of the first things that you have to face when you are making a project no matter if it's big or it's small is the architecture or the structure. If the project is small you can keep on the proposed MVC of Django with the back and the front coupled but a bigger project will face bigger and harder challenges and you need stronger foundations. In Gear Repsol for instance we started developing this website and six months later we were asked to expose an API for third-party applications, for mobile applications, a mobile version of the map and several other integrations and we weren't prepared. We had some API modules but it wasn't really an API from one side and we struggled to update to the new requirements. So the lesson that we learned the hard way was completely separate API and front projects because API is usually stable or maybe more stable and the front projects change every day and you can add lots of new clients or new applications consuming the API. So different life cycles completely separate. As well as being prepared on the big picture it's also necessary to be prepared in smaller modules. For instance we knew we had to make a roadmap but we didn't know the map provider because the customer hadn't made its mind. We thought it was Google Maps but it wasn't sure. So we prepared an isolated module which was talking to our inner code and this module was the only in charge to talk to the server and third-time map provider. Finally we integrated with Google Maps and we didn't change but Google Maps changed the API two months later after the integration because they can do this and we only needed to change in this isolated module to be like in very with very few efforts we were prepared to update the requirements of Google API. So be ready for changes but don't over engineer. It's difficult to know when are being prepared and when are over engineering and some rules may be useful and the rule that is the lesson three is refactor and abstract the second time you have the chance. That means that if you are working within a method for instance and you have to parse a string you can parse the string within the method but the second time that in other part of the code you need to parse a string remember that code that you did and abstract this there's this new functionality in a utils isolated and call the new function the new method for everywhere in the code. That way you will refactor the second time and not before you need it. Okay. So before continuing let's carry out a very little survey. Please raise your hands. Those of you who have heard about the laws of thermodynamics. Okay smart people of course you are right in my talk. Now raise your hand if you know by heart the fifth law of thermodynamics. Yeah I was expecting that it's pretty common to forget it and it's a very important law which says sooner rather than later you will make a bug. It's not me. It's thermodynamics. It must be true but being that true the next lesson is like quite easy because you have to test, test, test. If you have a big application a big project you need to intensively test the code because you will need to be sure that your new code won't break the behavior of the application and you will need to help your colleagues to add new features without breaking your code your precious code. So you need to test whenever you are adding code and in the result for instance even if all of us knew that you have to test we wrote test but not many and we struggle because some part of the code made us fear and we didn't want to touch it. Only the person who originally wrote that code was in charge to change it because we didn't trust ourselves which is normal and in Python and Django you have lots of utils and libraries and tools to test. You need to know about testing and really, really test your application. In Django when you are developing with Django and this is like specifically for Django and Python you need to know about the strongest point which is the community. If you go to Jfab you are going to find like thousands of projects with Python or Django tags. So when you are when you need something for your application you have to think that maybe probably someone has solved before you and probably that person has done test and documentation and has done better than you are going to do and you don't need to put effort that. So that community helps us to meet one important part of the point in the web development which is don't reinvent the wheel. But sometimes the library that you find doesn't exactly do what you want or maybe it's too big and it's a very big dependency to add into your project and you just need a very little thing or maybe you need the whole control of what's happening. So in that cases the advice that I can give to you is that you have to know the internals and don't be afraid to do yourself. In Python and Django it's open source so you can hack almost everything and you can add new features and you can improve and extend as you want. So no Python, no Django and don't be afraid to do yourself. As well as these big headers there are other good practices maybe they are not so technical but they are also very, very important. For instance if you have been working like for a day in web development you will know this lesson but nevertheless it's important to say it so use a distributed source control manager. You have Git and Mercurial. Mercurial is made with Python, very cool, you can hack it. Subversion is not distributed, subversion is from the past, don't use subversion, Git or Mercurial or other if I really don't know others but maybe. So with these kind of tools you will ensure your code to be breathable, you can go through the history of the changes, you can undo changes if you don't want to and when many people are committing into the same code base you need to use something like this. In GitHub we knew this but we introduced a new procedure which is like the lesson eight, use pull request as the basic procedure to have features to master. That means that when we were starting a new feature we created new brands from master and then we developed the code, we developed the test and the documentation and when we were comfortable with the result we made a pull request and we made a pull request and another colleague was in charge of reviewing this code and testing it and maybe suggesting some improvements. That way all the code which was entering in master was reviewed by two people, the original developer and the reviewer and at first we thought that it would make like time overhead, we were losing our time and it was all the contrary we were making less bugs from the very beginning, it was a very, very good new procedure to add into our team and it's very recommended. One often overlooked issue is documentation, maybe we think that we code so, so well that we don't need to document our code, well no, it's not that. You need to document your code as much as you can and do it okay, do it well. In Gear Absol for instance we had two documentations, one for the internal team concerning the reasons of the some decisions of design, concerning the versions, the deployment, concerning the code and we had a different repository for the API documentation which was meant for those developers who were consuming the API and they only, for them only responses and status matter and everything else they don't mind to. So we made those two documentations and for the new features concerning the API we made also pull requests and we used this kind of documentation driven development which meant that for any new feature concerning the API we made first a pull request into the API documentation repository and the colleague in charge to review it was someone of the front team because they were the first people using the API. So if they, when we agree with this API documentation the back team could start developing and it's like the designed by contract but focused on a higher level of functionality. For us it really worked very, very good finally and it's very important. When you are working with Django, with Python or maybe with other open source tools you are using, you are taking advantage of the open source ecosystem. So those tons of libraries that make your code better and your life happier are part of the ecosystem and it's important that you should give something back to the system. So when you are developing make as much open source as you can. There are many, many ways to contribute. It's not necessary to be a top-notch engineer. You can report a bug, you can fix a bug, maybe create a tutorial, you can create a workshop, a fantastic workshop of Django Girls and you can of course release a library if you have done this. In VRF Sol we have, sorry, in Kledos we have contracts with our customers and in the contract they agree to release non-critical code. So whenever we are working and we need to, we discover some part that would be useful for others, we make that to be a library, release the library and then we use this code as an external library but it's available for anyone who wants to. And also in Kledos we have developed our own product so we are on our customers and we have released it from the very beginning. It's called Taiga Yo and it's made with Django and with Angular and we are very happy and very proud to share it with the community. But if you want to share it, if you want to give into the wild and if you want the community to contribute with you, you need to follow those steps and maybe other steps to make it easy and clear how to contribute. You need to make test, documentation and you make a beautiful API. You have to put some effort in making things good not only for performance but also to share with the community. These are the ten lessons that I can give to you. I learned during two-year project. Most of them apply to any big project but some of them apply specifically for Django because you need to know which are the good maintainers, which are the obsolete libraries and it's important to know. So I think that that's it and do you have any questions? Any questions? So you mentioned that you agreed the documentation for the API up front before you started coding. Did you then write the tests against that agreed API before you then started coding up the API or was that done in parallel or how did that work? Because it would be really nice to be able to agree the API and then you know what the test should be. So you are asking if we wrote tests on the documentation? Yes. So once you've agreed in the documentation what the API is going to be, did you then write tests that then obviously fail because you haven't written that thing yet and then write the actual code? About documentation we have tried several things but in the end what we made is just writing the documentation by hand because we didn't use any specific tool. The tests were about the implementation. So when we started implementing we made the test maybe TDD for that specific part but the synchronizing the API documentation with the implementation is a matter of the human effort. Yeah, sure. Okay, thanks. A question I have myself. You said that your customer agreed for you to open source parts of the project. Was that something they were happy to do or did they need convincing? If they are happy. Yeah, were they happy when you suggested open source? They are happy because at first time maybe they didn't know us but when we are like signing the contract they know us and we have like convinced them that it's very cool to be part of that and of course it's mandatory for the first thing so it's not a decision that we share with the customer but they are not like forced. They feel comfortable with the decision and they feel very happy because we have shown them that I am not implementing my own API. I am using Django REST framework. So if they don't want to release, I will write my own API which is crazy. So they are happy and sometimes they go saying with other colleagues like oh, we are in this open source technology and so on. So they are quite of happy. Thanks. Any other questions? Thanks for that. So you said you were the people behind Tyga IO? Yeah. Just a question. I haven't checked in a while but last time I seen it there was no live demo. Do you have live demo already? Live demo of Tyga? Yeah. We have Tyga in Tyga IO. Tyga.io is the web you can sign up and you can play with it and you can start it on premise so be welcome. Thanks. And if you want a specific demo we can talk later. Okay. I can do something in that part. You mentioned separation of frontend and API. So how do you separate it and at the same time how do you make sure that you are actually using that API or do you write in a different access methods and different code paths for your frontend? In our case the front team and the back team we are together so we agree even the architecture from the very beginning but nevertheless in the project we have those mockups which are the design and the customer agrees the functionality. So as long as we are exposing the data and the front is consuming the data and the customer is agreeing with the code we are like the only proof that we can take is the approval of the customer. I think is that what you are asking? Yeah. Okay. Any more questions? No? Okay. Thanks very much, Yamada.