 And my friends can testify that how nervous I am, because it's my first meetup talk. So thank you, thank you. So yeah, but when I was seeing his talk, like I feel nervous even watching his video. I cannot imagine how I would get so nervous even to drive in the US, I cannot imagine how I can fly. So hopefully it is less nervous now for me. So introduction for myself, my name is Meng Yi-Yuan. I am a solutions engineer for WhatsApp at Facebook. And you can follow me on Twitter, because I tweet a lot of cat chips there. But enough about myself, because I think there are something very important that I want to pitch you about today. And the topic is about how learning in DevOps will help you to become a better developer. The reason is that, especially if you consider the term DevOps, there are two parts of it. And I believe everyone here is familiar with the developer part. But for the operations part, I have seen so many developers that they just don't care. And what I want to show you today is that why you should care about it and why it can help you to become a better developer. So before going into all the details, there is this book called Start With Why. You can basically read the book by reading the title itself, in which it figures this golden cycle. It argues that before starting to do anything, before worrying about how you should do something or what to do about something, the first thing you should ask yourself is why you should do it. So applying the same concept here, why do we want to learn DevOps? Because it helps you to become a better developer. But before that, we should first understand what is DevOps. So I was at DevOps Day Singapore two years ago. And almost every speaker there start with some definition of DevOps at the beginning of their talks. And almost none of them is identical. So it is about one of the technology terms that has the most variable definitions. I won't attempt to define it myself. I only borrow it from the experts. So one of the people that I really, really admire is Jess Humble, who is the author of Continuous Delivery, which I will go more deeper into later in the talk. He defines DevOps as a cross-disciplinary community of practices dedicated to the study of building, evolving, and operating rapidly changing resilient systems at scale. So it is even a mouthful for me to read. That's why I am me. And Jess Humble is the one who came up with this concept. Some other concepts that I take from the agile admin, if you'll Google what is DevOps, they are the first term to come up. DevOps refers to the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production. So both of, if we extract the common points from these two definitions, basically, they all argue that DevOps is a set of practices where the engineers will practice. And it helps you to build scalable resilient systems at production. But I think we are, again, going too far. We are diving into what DevOps is already, but we should start with why. So we take out the golden circle again, and we should ask first why DevOps at all. Why does the industry need DevOps in the first place? So it's worth to look at days before DevOps. So there is this wall between the development team and the operations team. Basically, the developers want to do all the shiny new stuff, and they want to push out things as fast as possible. On the other side of the wall, the operations team are the ones who need to maintain and manage the application in production, and they are the ones who got paged at 2AM on a Saturday. That's why there is this conflict between what the developer wants and what the operations team wants. Plus, developers normally use tools or frameworks that the operations team doesn't need, doesn't use. So this mismatch of knowledge and tools that they use have combined into this result where that the developer will come up with some new codes, and they will just throw over the wall to the operations team with little to no documentation. Then the operations team cannot run anything because they don't know how. And then they throw back to the development team, and the developers will say, it works on my machine. How come you cannot get it to work? Those are the days before DevOps. IT is like we want to think that we are ahead of the industry, but actually, if we go back into history, we would see that the industry leads us into why we want to do agile and DevOps. So I bring up Jess Humboldt again, and he argues that if something hurts, you should do it more often. Instead of trying to avoid the pain, you should do more often so that you are forced to improve your process and make the process less horrible. So just a short history of DevOps, although there are a lot of words there. Basically, the entire idea of agile and lean manufacturing starts in the manufacturing fields by Toyota in pre-2007. Then in 2008 and 2009, finally, the developer cycle starts to catch up with this trend. And the term DevOps was coded in the history book in 2009. Then since 2014, the Dora, which is DevOps Research and Assessment Organization, they start to come up with this state of DevOps report each year. They give out survey questions to engineers, like basically whoever works in the IT field, they can be developers, they can be operations, they can be system admins or DevOps teams, or they can be executives as well. They will give up this service, and they will assess all the business based on how well they are performing in terms of software delivery performance based on these four criteria. Deployment frequency, the time for change is time to restore service, and change filler rate. Then they will categorize all the participating businesses into four categories. Previously, there are only three, but last year they had an elite group as well. And the result from the report in 2018 shows that there are only like 7% out of all the participating businesses that are categorized to have elite status. And the result and the difference between the elite and the low performers are just stunning. You can see all the four criteria mentioned just now, they are significantly higher. And for failure possibility for a change to fail, they are 1 7th as likely to fail a change. And even from an organizational perspective, for the elite performers who adopt DevOps, they are seeing much more likely to meet or exceed organization performance. So there are already many talks that argues about the value of adopting DevOps at the organization level. I don't want to repeat too much about it, because today what I really want to focus on is why you should care about DevOps and what DevOps will benefit for you. So going back to the title, how can learning DevOps help you or help me to become a better developer? First of all, DevOps makes you to write better code, because if you want to have continuous delivery of your code, you will have automated tests. And to make your code testable and automation easy, you have to write clean code and loosely coupled architecture. That is some discipline that you will never remember if you don't have continuous delivery. Also, you need to produce good quality logs for better application monitoring. For developers, sometimes we don't care so much about logs. We just troubleshoot with console log and then remove them before we push into source code. But then if you really operate your code in production, you will find out that you almost have no insights into what the customer is doing. If they run into some errors, you will only see the error for the most part, and you do not know what they are doing before that. So adopting DevOps also require you to embed good quality logs into your application. And the last point that I want to argue based on my experience is that if you are a developer who works on DevOps, it is very likely that you will be pairing or working with the most senior developer on the team. This is exactly because very few developers care about DevOps. And if someone is going to work on DevOps related stuff, it will be the most senior people. And if you are the developer who cares about DevOps as well as they are, you will have a very good chance to be pairing with them and learning so much from them. It is very good for your career as well. Then on the business side, DevOps doesn't, I think, to become a good developer. It is not like you, it is not enough for you to only write good codes. It is also, it is better if you understand the business that you are working for. So DevOps helps you with that because you understand that software is not done when you are code complete or not even QA certified. Software is never complete, per se, because they will live forever in production. And they will iterate, they will deteriorate, and you have to manage as an entire lifecycle. And then you will be able to witness the software to deliver business values because software is only starting to generate business values when it is in production. And if you assume something will help your business, you will only see the real results after it is deployed and being used by the customers. And then you can iterate whether my assumption is right if it is not what I should do about it. Yeah, this is the point that I was talking about. So hopefully, I have sparked some curiosity in you and some interest about learning DevOps. And how do we get started? Just like any other software development knowledge, DevOps is no different. You learn, you make, and then you iterate this process. This also that I also like really much. I don't know how many of you know Terraform, but he has this book about Terraform. But he also recently had this book about how you can build a startup as a programmer. In which book he said software is a living and breathing thing that grows and evolves continuously. I would think that our skills as a developer is a living and breathing thing that needs to grow and evolve continuously as well. The parts to become a better developer, as I said, you learn, you make, and you iterate. So I will start. I will present my slides also in this order. I will present something that I have learned from and what you can do to start going. And then to go deeper into something that you can learn from, then to go deeper into something that you can implement. So starting from the most basic ones, something that you can learn or you can read is this book that I really, really like it. Because it is a novel which is very rare in the technical books field. It's about this really traditional business and how this shiny new IT project is way over budget and way behind in schedule. And the entire book is talking about how DevOps and agile practices basically turn this entire business who is about to collapse because of this new shining project is fully, how it was turned around by adopting agile and DevOps and then achieve better business results. So if you want to really get started doing DevOps, actually just now I was also talking to someone in the audience. It doesn't really matter that much what to you get started with. You heard about all these Stalker Kubernetes stuff. Actually, I would argue that they are not the most important thing. It's more about the ideas you want to embed into your project. So to start with, you should use version control. Also, although this looks really rudimentary, but this actually refers to not only your source code, but also all your configuration management script or your data migration script, everything that you need to rebuild an application in a certain environment, you should put it into version control except for your password. And you should use trunk-based development whenever possible. This is especially important when you are working with teams that has a lot other developers. Sometimes you are working on this big, big feature and you are very afraid to push to trunk, which is like the master branch because you will fear that I am committing something to production that I am not comfortable with. But if you are living on a feature branch for more than a few days, you will find out whenever you want to merge back, whatever you assume is there is no longer there. Whatever you assume the file path is there, it's going to change. It will be such a huge pain so that you should use trunk-based development and keep your changes small and manageable all the time. And you should also set up some basic continuous integration pipeline in which upon each check-in of your source code you should run automated unit tests. And you should generate an artifact from each build to be used later in the stage. So now you have some experience making something related to DevOps. You want to go deeper because you run into all these problems about how I should manage configurations over different environments. How should I manage my data? How should I write tests? So some other books that I find really useful, the DevOps Handbook. It is a very practical handbook for you to learn all the best practices basically in DevOps world as well as continuous delivery, which I mentioned briefly just now. It is so good. I normally just open it before I sleep and just try to. It's a very thick book as well. But almost every sentence there is good. If you are confused about anything, how should I do this thing? Jess Humble has an answer for you in the book. And then this release it. It is specifically targeted to explain how running applications in production is so different from any other environments, not only your local host, but also a testing environment. Because the load is on an entirely different scale. And you normally will run your application among servers in maybe different availability zones. How do you make sure that something breaks in your system doesn't bring down the entire system? So all of these things are presented in case studies, like real life case studies. So now we are hopefully after reading all the books, you have more knowledge to go on to make more things. You should further build up the CI CD pipeline. Now we are adding in continuous delivery now. So you should automate the development of the same artifacts that we built from the last stage to each environment with different configurations. There is this concept of build once and deploy everywhere. Basically you will be always generating only one single artifact from each build. And you will use the same artifact to deploy to different environments. After each development, at least you should be sure that the system is up and running. This is not guaranteed if all of your unit tests pass. Because even if you don't see any errors after you deploy, there are times where you think, let's say it's a web service, you think the URL is up. And when you are trying to connect to it, it is not. So if you run into such issues very often, it is good for you. Instead of manually checking it every time after development, you should have some automated smoke tests to make sure that at least the system is up and running. Then you can go deeper into automating more acceptance tests, maybe with even some performance tests to make sure that your system not only is barely up and running, but also that it can fulfill the business requirements that your application provides. After the deployment, it is also important to monitor your system because your application still lives there in your production after you deploy. And it's crucial to monitor the health of the system at all times. You can configure most of the monitoring tools that you will find in market or in open source world today. You are able to configure it to send alerts when the metrics you are subscribing to go off track so that you don't need to be alerted by your customers. You can always find problems first and solve it hopefully before the customer realize. Just now I mentioned that DevOps, for me, not only helped me to become a better developer in writing better codes, but also helped me to learn more on the business side, or at least to be more interested on the business side. This is the book that I mentioned just now. I find it really, really helpful. It contains a lot of DevOps practices as well. Not surprisingly, because DevOps is really, really crucial for building a successful startup. You cannot afford your system keeps going down when you are so busy marketing your products. And then when a huge group of users come to your website, it goes down. That is not good for your product, for a startup. And then this very famous, the lean startup. It also introduces the lean manufacturing concept that I mentioned at the start of the talk. So on the business side, actually, the DevOps can also help by monitoring and measure business metrics. And by monitoring and gather data about these business metrics can also help you to make business decisions. Let's say if you want to define a new registration flow, you think you are making the registration floor easier or prettier so that the customers will like it better. But actually, how do you tell? It is not by guessing. It is better that you can measure it. You can implement what we call AB testing so that some users that comes to the registration flow, you can send them to the OAT flow. But some you can direct them to the new flow to go through the registration. Then you can measure, let's say, how long it takes for them to register how many errors the users ran into. By just being able to measure these business metrics, you are able to determine whether your changes actually achieve the business results that you think it will. And then DevOps is not only about automation. It's not only about writing scripts. It's also about a kind of culture. And one of the most important culture that DevOps foster is also the learning culture. You can learn by sharing what I do today. I know I am still super nervous and was rushing through a lot of things that I was planning to say during my rehearsal. But it is a good step that you can come up here and share something that you learned during the journey. So I was told every presentation should end with a story. This could be a long one, but I just want to go through my stumble into DevOps. So some backgrounds. I graduated from NTU, but not with a CS degree. It was an electrical degree. So when I was in school, I was doing some development projects. But I don't know what DevOps is. I don't know how to put my source code into servers so that I can serve up a web page. I don't even know how I get my final year project completed. So after that, I joined MNC, the multinational company, as an internal IT team, which we are responsible for managing business intelligence platforms like Tableau and those kind of platforms for internal teams. At that time, because of the company that I was in, we are purely using Windows servers. And this was before Microsoft starts to become cool and acquire GitHub and have VS code. So all the Windows server that I interacted with, I don't know how automatable they are, but I was just clicking around. And every time if we want to spin up new machines to host these business intelligence platforms, I will do manual provision and configuration so that a new environment can be up and running. This is really horrible, because every time something doesn't work, you don't know why it doesn't work. I ended up manually comparing why in another environment it works, but not in this new environment. The best that I have done development-wise, it was a C-sharp project. And it was also before .NET Core. And every deployment needs to be executed from Visual Studio without code. So you push a button and input something. I wasn't involved because there is another senior developer as I said, because nobody knows how to do it. It has to be the most senior ones who know how to deploy things. And it was all manual. The best I have done is to write some PowerShell scripts to automate something, but every time I have to manually change something. And hopefully I don't make a typo somewhere and break the whole system. So that was that. Then I started to get fed up with doing things manually and comparing configurations. And I feel I should improve my development skills in any way as possible. So I joined a consulting firm where I really started to learn about what DevOps is about by pairing with senior developers again. For all the three projects that I have done in the company, I somehow I all ended up automating their deployments, even doing some contract testing. I will share the slides later. And you can find what contract testing is about. We are able to automate some acceptance tests in one of the projects by managing test data, as well as, well, we even go deep into automated environments provisioning with Terraform and stuff. It was really fun. I was really, really unnovous. I didn't, before joining the second company, I didn't even use Mac or Linux Box before. I was, in my first project, I was so embarrassed that, like, let's say I just know LHS properly. That's the only command that I know. But then being embedded into all of these DevOps gives me the exposure of not only writing application codes, but actually how you can deploy your code into Redaction. Then I joined Facebook. And so my role is solutions engineer. And I'm very fortunate, actually, to be assigned to the WhatsApp team. The main product that we are working with is called the WhatsApp Business API, which they launched just last year. And because WhatsApp is really, really, really end-to-end encrypted, and when you are using it from your phone, the WhatsApp app on your phone is actually doing the encryption and decryption of the messages. So for a business to use the API to send WhatsApp messages to their customers, to encrypt the message, the business should have something like the app on your phone, but we cannot use our phone. So what we do is that we put the software in our Docker container, and we require all the business to deploy these Docker containers in their data center before they can use the API. And you can just imagine how many businesses are not aware of how to do this properly, and we have run into so many problems. But this is where I feel really excited because now it's the thing that I am very familiar with and very excited about. So even being in a big company, where, especially for Facebook, we have dedicated release management team, infra team, like multiple teams, which I am not part of, you can still be the subject matter expert on your team, because, again, so few developers care about DevOps if you are the one who understands what Docker is, what Docker compose is, what Kubernetes means, it is such a good plus even from your career perspective. And another thing that I worked on first during my time with WhatsApp is that we have this tool that we built in Python. We have like three to four developers work on it at the same time. Previously, we want to move fast, right? That was the motto of the company. So we start, we are not using the standard source control system of the company. We are just using GitHub, not GitHub, just Git. And then a lot of times it happens where someone checks in some code and then when I check out, it doesn't work anymore. It breaks it because we don't have any automated tests. We just check in after the assumption that it works on my machine, it should work on others, but this doesn't work. So there is this long time where I spent just to migrate this existing repo into our existing source control system that the entire company uses, which brings in like all the linters, all the automating test system and it's just a life quality improver, you know. So what do you should do? If your company doesn't adopt any develop practices, please be the advocate, show the value and start doing something today. And even if your company is like on the scale of Facebook or WhatsApp, you can, where they have already adopted all the develop practices, but you can still improve on it, fix something, improve, put more tests in your code and just generally make things better. And then for your set projects, it is a very good place to start. The most common that I have heard about DevOps is that it can be really difficult to get started if you are working alone, but if you are working with your set projects, you can just start by what I said just now, use version control and try to run automated tests after each check-in and deploy it somewhere, like all automated, just see how fantastic it is and then you will always want to do this and never back. So software is a craft and never stop learning. That's all for my talks.