 Okay, everybody's here. We have a final speaker for the day. It's Ricardo Banffi I was going to talk about lessons learned after 190 million lessons served. Good afternoon. Thank you for coming I'm here to talk about some lessons before anything else I have to say that I'm new to Udemy and Most of what I'll describe happened before I arrived What I'll talk about is therefore the work of other people and the lessons, however, I think they stand on their own Well, this is where we are now Udemy was founded in 2010 And in six years it grew to 11 million students 40,000 courses given by 20,000 instructors in 80 languages The 109 million lessons served number I used for the title was gathered from a table that tracks students watching our videos Lesson one is about preserving our culture as we grow We're a learning company and learning is in our DNA For any healthy organization Preserving its culture is essential to its well-being What I'll talk about is about the engineering team or the teams have different Different things they do Well, we are very serious about onboarding onboarding new employees The idea is to get a new hire From a new hire to a productive engineer in the shortest possible time We use extensive automation. We have standardized Development environments Development environment that's reasonably flexible and if you really want to you are able to deploy our website on your first day I did that in the first week Even I can do it. We learn and teach Ourselves using our own tools There are Udemy courses about our applications from the user perspective from the back end perspective That are given by our own team So you can hear about the application as it is described by the people who created it We also communicate a lot We have hundreds of people in three continents and We spend 11 time zones We all hang out in Slack We use hip chat until last like yesterday literally We just moved We also use scrum We have the usual stand-ups grooming sessions We also have hangouts Hangouts We can do so we conduct all hands meetings on hangouts or other solutions if we have too many people Every meeting room is capable of joining a hangout. We have big screen camera good microphone So we can join we have meetings. We can have easily meetings that span more than one team in one place Well, like I said before we use automation a lot We automate the quality checks we We can't commit ugly code Seriously, we can't commit code with one extra line. We can't commit code with missing spaces We can't push broken code That means get runs a basic set of tests before we are allowed to commit You're allowed to push to commit. It's flake eight. So still anyway and You can't merge if your code doesn't pass Very extensive set of tests. So you're absolutely sure that when you merge your code, you will not break anything Also We also use custom tools custom Django admin scripts custom Django admin commands To help prevent accidents So if you want to update a series of records a series of Objects in the data store, you will not be handed a sequel my sequel prompt In fact, it's very few people who have that kind of access Instead you'll have you'll do that through code. That's Reviewed by someone else That's known to be I Won't say bug free but it probably is less bug free than you typing sequel commands on the my sequel prompt Even though we do We go to such great lengths to prevent that some accidents happen I myself deleted a shared database on my fourth week fourth week the company I was out of the Dublin office where I'm based. I was in San Francisco and I Deleted one of our shared databases Not production environment development environment, but thanks to me the whole development team with the San Francisco office Which very large actually Had a mandatory coffee break Because no one could do anything Yeah, there's some room for improvement What we can do to that Well, we have the shared database. We're working now To have an individual database for everyone With just enough data so you can so that you can run your development with meaningful data We also constantly make improvements to our Development environment We add all the time we add code that makes it easier to develop we've had extra checks with add Some extra functionalities that were not there before Personally last week I think no last month I added a Bit of code that made some tree objects represent themselves in a nicer way on the terminal on the IPDB prompt It was a horrendously complicated logic and by that I had someone lessened my cognitive workload so that I with my Somewhat limited brain I could wrap it around a very complicated problem without having to worry about Imagining what the objects look like I could just print them on the screen and have Look into them The second thing the second lesson is that as you grow You start to you have to well all the time you should you should be measuring everything anyway When you grow you will start measuring measuring it in bulk So every application we have generates Collects metrics about themselves about itself. We collect those metrics in a huge Not Facebook huge but still pretty impressive scale For that we use stuff like sentry Datadog and chart.io This is not Although this is considered something important to us These applications are good enough. So you don't have to develop them again. These guys are good so Datadog Sentry for alerts Google Analytics and most of front-end is Constantly monitored of that Also, we have a huge user base with that comes the possibility of doing extensive tests and experiments Where you enable some feature and you can track on a B tests how the user base behaves Since you have a lot of people coming you'll have good quality data on that and We can actually know how something will impact us well before we enable it to the whole user base This way surprise are minimized Which is good. I like I don't like surprises That also allows us to have early warnings if something starts to If you see the distribution say of execution time the given function and you have a peak at like a hundred milliseconds And you start developing a peak at zero milliseconds. You know something is wrong so you can watch those distributions on a tool like Datadog and This data must be Timely we must receive it Soon after it's collected we can we must be able to see it immediately This large user base is all It's convenient because it generates a lot of data we can analyze But it also amplifies any trend very quickly and It's really not that good for you if you only know that What caused an outage of a week-long outage of a service? You don't want that you want to know if possible before Am I going too fast? No Yeah, I suspect you like this one It's about questioning and being a bit Willing to abandon an approach Let me say that they Netscape in this case Delayed a new version by three years. They did it by making the single worst Strategic mistake that any software company can make They decided to rewrite the code from scratch Okay This is Joe's Polski. Who knows Joe's Polski? Raise your hand Well for those who don't know him. He's a pretty clever guy He created a lot of stuff used daily Like Trello or Stack Overflow. Who's a Stack Overflow here? Come on No, I don't believe you. No, no Everyone should raise hands that Okay He said it in an article This things you should never do that everyone should read even if you want to disagree with him later Told you you'd like it Yes, we moved from PHP We had a large PHP application built on a custom made framework created by designed internally by our founders Who here has has built their own framework who thought at that time it was a good idea Who regrets that now Okay, we're good This is a problem because new hires has to be trained on a framework they have never seen and It takes a long time It ties an expensive resource it ties an engineer that's already trained on that And the engineer will be tied and not building useful things useful things for the business useful features Things users are demanding That's specific always had the very little tests With the lack of tests maintenance becomes walking on the minefield The fear of breaking stuff and not seeing it until it goes into production or worse yet until the corrupt user data Will drive you to adopt a very careful approach You will always you end up opting for safe changes Instead of using it instead of writing some more maintainable More elegant more performant Code you may even be tempted to replicate side effects of a previous implementation out of fear you'll break something you are not seeing and Obviously this makes development slower and more expensive so what did they do and I say they because I arrived right after the The migration was finished, so I didn't even see that happening. I just such a shame. I'd love to see that I'd love to be able to delete some PHP code. So what did they do? No, they didn't jump for the first technology or the technology. They love the most They actually created a project and evaluated they evaluated the various options they did a small Proof of concept project with each of those and with what they learned they Guided the decision they looked into How easy it was to develop for that platform what they knew How well the platform encourages good practices and How easy it is it would be to onboard New engineers on them. Yes. This is a serious project. It's not a Lot of resources was put behind it They set deadlines they agreed on goals They did that as if the company dependent on it because actually it did Well, I know you probably have guessed which technology they chose because I wouldn't be here talking about it I wouldn't be on nude con for something so other conference telling stuff instead of here Well, what you didn't you got what? And what beautiful environment I am I have the luxury of working with Well, we got a modern Django based application And modern it runs in Python 3.5 and Django 1.8. We are considering the move to Django 1.9 It also allows us to onboard engineers much quicker Because Django is an opinionated framework and you don't have to teach every decision. We made every decision You didn't made a lot of decisions already baked in the platform We have 88 percent of test coverage That's not perfect. We can do we probably will improve that One of the things is that we can't commit code We can't merge code if with the merge the code coverage goes below a certain threshold I have cursed this decision a couple times myself in the end. It's for good with all that We had also a lot of new features We had a more consistent code base Which is what opinionated frameworks frameworks are for And well, there's some of course as anything that was migrated There is some scar tissue where PHP and Django coexisted some table names are a bit odd some field names are a bit interesting and Do we have some special few types and some parts when we have to fight to fight Django's opinions and let me tell you Django is very persistent in fighting back Who had here to fight back? Now you laughed come on someone raise a hand Okay, and what we did we learn from that? Well the immigration took a while to what's two years Two years with 30% of the engineering team dedicated to it It was done incrementally feature by feature and there's a motivation thing on that The the team involved the team responsible for the migration Track the progress by counting the lines. They actually delete it. This is why I'm wearing this t-shirt They call themselves the deletionators We've cried deserved one I'm I borrowed the t-shirt. I didn't delete any PHP line. So All the time the website was up and running and It was as you would browse the pages you would see an high a hybrid PHP Django application Showing your your page showing the data you were used to also Since everything was changing Mindset will set will mindsets would have to change. So that's where the Automate automated checks come from so instead of ending up with an untested code base They enforced some things that they considered mistakes. They learned from the PHP code base and they worked Against the impulse to allow the Django code base to be like that It's also final lesson here. This is about The importance of the mission as I mentioned in start you we are a learning company We exist to help anyone learn anything That's quite a goal This also means that anyone can teach something to anyone else We all have things we like to talk about we all have passions. We all all have Stuff that he would like to teach other people if we could or There are stuff we'd like to know more if only we could find a person who could teach us What is the impact about one third of the Udemy students are starting a growing their own business using the Knowledge they acquired Through the platform for that We also have lots of people who became full-time instructor instructors who basically changed careers from being Something to teaching that's something they love to do if you visit or stand you'll see a couple sheets of stories of how Udemy changed the lives of those people and I'll tell one of those stories not one that's on the that sheet those paper sheets Well I'll tell the story as it was told to me by my colleague Aaron who actually knows this smiling nice guy He's Lynn Smith he Had a lifetime career as a copywriter He wrote ads for companies like Vodafone Lloyd's BBC IBM And after this lifetime career he decided it was time to retire and To keep himself busy as his career as he winded down his company his career He got involved in Udemy And he decided to start I can't start teaching people to write proper copy well It turns out he Probably didn't retire because as of now he has taught more than 56,000 students I myself was very impressed with his number because I was once a teacher and My largest class was like 50 student 50 60 students For a course that took 20 classes 20 weekly classes if I were to get close to what his this guy did after he retired It would take me 500 years To do that This is a 10x guy 500x guy probably When I read the stories when I read the messages our internal emails send Forward that our students sent I see our mission is very very important In the end everything we do everything we do at Udemy everything we all Do here at Europe Python is about the people is About the community it's about people it's about teaching it's about learning and sharing. This is what we do Thank you. Oh one more thing if you in case you buy a course if you use the import this for zero Code you'll get a 40% discount any questions, okay give a bigger, okay Thank you, and I know nothing about Django, but you said that you have to fight the framework back An example like I know nothing Django's opinionated framework, so it assumes you want to do stuff in a certain specific way I know it's no longer the case but a good example I can bring up is from I Think a long time ago, and I met that situation. I wanted to I was doing an image bank for a portal and We wanted to save the images the originals in a way that they would be deduplicated with Which meant we would change the use the file name that was uploaded to a different one And they would play that they would place it on a different folder than what Django wanted to do It took me like one week to build the application and Three weeks to build the logic to allow the the application to save the files Where we wanted them to and it was really really ugly code at that anyone else Hi, you said that Django and All the PHP code coexist at the same time, right? You you probably didn't just Flip the switch So can you explain this a bit more like you probably couldn't use internal Django road system and other features? like how did you manage to Do this that the boat for systems coexist? Django and PHP share the same database with the same Data that's why we have some strange table names in some parts of some other parts of the system The URL routing was done both on the web server and the Django application So some some of the URLs would direct the PHP application some Sends you to the Django application Is that what you wanted? Okay, so after you launch fully Django, then you had to do some cleanup the code Yes, you basically hard-coded some of the URLs and yeah as the as the PHP code was being removed the rules that sent to it the code itself it was deleted from the configuration Any more questions? Sorry, it's not planned, but I can clarify the routing what we did is we had an engine X front-end low banter And we had roots custom roots a giant routing table in there That was selectively route traffic to the Django app or let it route back to the PHP app We just slowly deleted those roots and that more and more roots fall through to the Django app That's basically how we flipped over without doing a big bang release So things went wrong. We could just simply add the route back in and went back to the PHP app again Okay, thank you for that. So I hope to see you all at the lighting talks. Let's thank Ricardo