 For those of you who don't know me I'll give you a brief introduction about myself I am a lead web developer and DevOps engineer at Unleash Technologies and my primary coding language is PHP I do a lot in PHP. I've been doing it for many years now You might know me from the PHP League for my common mark library Or for my PHP 7 upgrade guide e-book, but I don't do just PHP. Although that is my Primary language. I also have experience with JavaScript Java C sharp Python and a few other languages Now this talk today is about debugging and the concept of debugging and the topics I'll cover apply equally to every language But because we're here at triple con and triple is powered by PHP and PHP is my personal favorite language You'll see a lot of these examples here will be in PHP But you can take these with you as you go and learn new languages So quick overview of where we're going today We're going to start off by talking about the importance of debugging Why are we spending an hour of our time today talking about this? Why is debugging an important skill to have? From there, I'll break down a five-step debugging process that I use and that I find to be really helpful And hopefully that can be helpful for you as well I'll also share some different tools and techniques I use when debugging certain kinds of situations and Then if we have time I'll open it up for any questions you may have So let's talk about debugging if you had to describe the word debugging or if you had to describe the process of debugging Using one word. What's a word that might come to mind? You might say debugging is annoying Debugging is frustrating debugging is time-consuming. It's hell. It's tedious yep all great answers me personally I enjoy debugging I enjoy puzzles so I might say debugging is fun it can be fun More than that. I think that debugging is important. In fact, I'd argue that debugging is the single most important skill in programming Now that is a very bold statement. Let me try and back that up with some data Let's say we wanted to break down the amount of time we spend during a typical day Where do we spend that time? We might say well, we spend two-thirds of our time coding adding that new feature It may be a third of our time planning updating task lists going to meetings things like that checking email But do we really spend two-thirds of our time coding and adding in these brand new features? According to the mythical man month the answer is no we actually spend up to half of our time Testing and debugging our code if you're going out why that code didn't run adding tests to make sure that the code works as it's supposed to So if we're spending up to half of our time on a daily basis Testing and debugging then debugging is a skill that it's really important to become proficient at so that's what we'll be talking about today Now whenever I give a talk I like to put a definition to what I'm talking about So I stole this right off of Wikipedia It says that debugging is the process of finding and resolving bugs or defects that prevent correct operation of computer software or a system And you'll notice I highlighted the word process because debugging really is a process It's not something that you should do haphazardly. It's not something you should actually Accidentally stumble into it's a process Process really is the foundation of effective debugging You need to have a solid process in place when you're presented with a problem to know, okay, I have this problem What do I do next? What's my next step? How do I go from problem to solution? So we want to have this solid process in place and once we have that we're going to use that every time We're faced with some issue the more and more we go through that process the more and more experience will gain We'll gain experience as developers with the languages and frameworks that we're using We'll gain experience with our IDE and debugging tools and gain experience with the code base. We're working in as well So the more and more we debug issues and the more we go through that process and the more experience we build up We're eventually going to develop our intuition this ability to kind of instantly know what kind of issue you're dealing with when it's presented to you You might come across an error like connection timed out and the first time you see connection timed out It's kind of scary. You don't know what it's about But by the time you've solved 10 connection time out issues, you'll know, oh, that's a networking problem There might be a firewall up some port is not open. Maybe my sequel isn't running And that's that kind of intuition you want to develop that ability to immediately know, okay I see this type of issue. I know where I need to look. I know what kind of tests I need to run to get this resolved So in order to develop that intuition all you need to do is just focus on the process and that experience and that Intuition will come with time So let's talk a little bit about that debugging process I'd actually like to start by talking about the process that a more junior developer would go through when they're faced with an issue So someone who's kind of new to programming doesn't have a lot of experience What are they going to do when they're faced with some issue that they need to debug? Well chances are they're going to try the usual steps They're going to clear caches do a composer install chmod everything to seven seven seven and This may fix their issue these are valid solutions sometimes Well, maybe not the last one, but the other two can be valid solutions sometimes And so they're going to try those and maybe they'll work Maybe they don't if they do work the junior developer will say, okay, great I solved the problem and then move on with their day Now there's nothing wrong with trying these But if you are a junior developer and you're trying those usual steps take a moment once you find that it works to ask yourself Why did that work? Why did I need to clear the cache for that bug to go away? Why did I need to run composer install? Why did I need to check the permissions of the file? So take a moment think through why that worked Don't just run the usual steps because they're usual steps really think through it and understand why that worked Something else a junior developer will typically do is to ask someone else Maybe they'll ask a co-worker. Maybe they'll ask Google or Stack Overflow and they'll get an answer Again, that's awesome. You should definitely do that But if you do that, don't just apply that solution and move on with your day. Think about why did that? Particular answer solve my problem. Why that one not this other thing that I found or this other approach that I tried Why wasn't this one the one that solved my issue? Something else that I think junior developers tend to fall prey to is what's called the XY problem They're faced with problem X and they think to themselves. Okay. How do I solve X and Try and solve it try and solve it and they're just not really getting anywhere So what they'll do instead is they'll say okay Well solution Y might work and we'll go on Stack Overflow and ask how do I do why? the problem With this XY problem is if you fall prey to this you are giving up on finding that proper solution You're looking for a workaround instead of trying to find what is the actual solution to my current problem? So you really want to try and focus on solving problem X even if it's difficult just be persistent As you'll see later in my slides I persistence really is key to developing your experience Even if it is difficult try and stick it out and try and solve problem X without resorting to a workaround Because a workaround or an incorrect solution can cause problems later And I'd like to actually share a personal story of when I was a junior developer and I fell prey to the XY problem So many years ago. I got my first big Magento project Magento is an open-source e-commerce solution built on PHP It's something like two million lines of code and XML mostly XML. Unfortunately It's a beast to work with Very high learning curve and for me as a junior developer It was very challenging for me starting off trying to do some complex things and the first client I had was a local restaurant that ships seafood overnight. You can go on their website You can order crab cakes. They're from Maryland. So of course they have crab cakes And you can get those shipped overnight on ice packs right to your door, which is kind of awesome Awesome concept. We get to build this custom thing in Magento So one of the things that they needed was the ability to print out FedEx shipping labels directly from the website So the customer pays for the order they can go in the back end click a button Generate a PDF with the shipping label that they slap on the box and send it on its way Now Magento didn't come with that functionality back then I think it might now But it didn't back then so I needed to implement that and I figured it out. It was awesome My first big integration and it worked and it continued working for many many years Fast forward a few years We had the second problem with this website this website was originally designed for IE 8 So all the JavaScript that Magento had all the JavaScript we wrote was for IE 8, but they upgraded to IE 9 and IE 9 just was breaking the entire back end. So I thought to myself, okay, I have this problem problem X I need to make this work in IE 9 How do I do this? It was kind of challenging. I didn't really know I found this solution online Maybe I can add this meta tag to the page and make the browser behave like IE 8 And this is kind of a valid solution However, I didn't know how to add this to every page in the admin area I didn't realize, you know, there's this base template. I can pop it into you or there's this really good way of doing things So I said, okay, if I need to get this on every page Maybe I can hook into this event that fires at the end of the request What I'll do is I'll use a regular expression I'll look for the head tag on the page and I'll just do a little find and replace and just pop this in right Before the closing head tag So this is definitely a workaround not the proper solution, but it worked. It solved my problem I was able to get the site working in IE 8. This was answered on the page. Awesome Until a few months later when those shipping label PDFs stopped working out of nowhere They suddenly became corrupted. We hadn't touched the code in a couple weeks and just broke. So that was really confusing FedEx web services was working fine. Everything looked fine Nothing changed. Why is it broken? Well, I tried opening the PDF on my computer and it said file corrupted So I said, okay, if the file is corrupted Let me actually open it up in like notepad or VIM and look at the actual bytes and see if anything in there looks funny When I did that I saw a bunch of base 85 encoded content Then I saw this in the middle here Now I was wondering okay, how did this HTML get into a PDF that doesn't make any sense How how did this code impact of this? Well, it turns out that base 85 encoding came contain the characters less than sign HEE AD So it the regular expression matched on that it saw the Closing greater than sign there and decided to insert that into my PDF So yeah long story short don't parse HTML with regular expressions You're going to have a bad time But more importantly solve problems the right way because even if you do find this nice little hack Or this nice little workaround or this other way of doing things it can cause these types of problems later So just make it easy on yourself. Try and find that proper solution Now there are some other things that I think we have Developers can sometimes fall into this we can sometimes fall into this form of magical thinking When we see an issue and we say to ourselves I don't know why this is happening or for some reason this broke This behavior doesn't make any sense to me and if you find yourself asking yourself these questions Realize that computers are logical machines They're not magical black boxes that do their own thing and have their own minds Computers are logical the code we run on them is logical computers only know two things zero one Off true false. Yes. No, there's no maybe bit. There's no sometimes bit the code is logical and If a bug is simply a defect in our code then a bug must also be logical And if a bug is logical it has a logical explanation that you can find if you're persistent Now I really like this quote from Nick Parlanti in debugging son He says the bug is not moving around in your code trying to trick or evade you It is just sitting in one place doing the wrong thing in the same way every time When you're trying to debug a problem and you're looking in this one Source code file the bug isn't you know jumping out of that file into a new file And then you go look over there and it jumps back. It's not doing that Again a bug is a defect in code the code is right there The code is logical if there is a logical explanation for it You just need to be persistent enough to find it and to identify what is the logical steps that it takes to replicate this issue Now when I'm faced with an issue the very first thing that I assume is that my code is the problem I almost never assume that my code is perfect and it's someone else's bug the reason being that You know 95% of the code I write is my code If I'm using a project like Drupal There are thousands tens of thousands of contributors using that project testing that out the chance that my website is broken because of a core Drupal Bug is extremely low More likely than not I put a bug in my code and I didn't know about it So if you know let's say 95% of the time that bug is my fault. Why would I want to spend my time? Looking at the Drupal core to figure out why my site is not working I should look at my code first because that's where the highest probability of that bug being is And if you're helping others to debug a problem in their code, I think you should also assume that Their code is the problem Now you don't have to be you know kind of snarky about it be like ha ha you suck at coding But again just game of percentages if there's a bug that just popped up and they're working on the code chances are they might have introduced it So if you're are helping Someone else with a problem take what they tell you with a grain of salt try and challenge any assumptions that they have made The only assumption that you should be making is that your code is the problem Don't assume that your code is perfect Don't assume that you've covered every edge case unless you have actually looked at the code and verified that it is indeed doing What you thought it would be doing So that kind of covers some of the philosophy of debugging some of the things you should watch out for some of the Assumptions you should and should not make what I'd like to do next is to actually share a five-step process that I use to debug And this is a five-step process that I didn't really realize I was doing it until I sat down one day and realized I have this logical process for debugging these logical issues And it kind of breaks down like this There are these kind of five steps that I go through and I'll go through these in a little more detail in the next slides But first I start off by gathering information about the issue figure out what's going on. What am I seeing? What is the behavior? After that I try and replicate the issue with a hundred percent certainty. So figure out what steps do I need to take to make the issue come up Once I've done that I'll go in and actually identify the culprit fix it Retest and then prevent it from happening again So let's talk about these in a little more detail The first step is to gather information and the most important thing you can do is to identify What is the expected behavior of your application versus the actual behavior? So I expect that when I download PDF the PDF is valid the actual behavior is that the PDF is corrupted for example Other good pieces of information you'll want to gather any error messages that you have Error messages are my best friend I think the nicest thing that error messages is that they're not randomly generated Anytime that you see an exception that has been thrown or an error that has occurred There's a good chance you can Google it and find a solution You can search the code base and see where that exception was thrown from That that can help you lead back lead you back to the source of that issue Even more helpful than error messages are stack traces a stack trace will tell you exactly what line of code Caused the issue and it'll tell you what line of code called that what line of code called that all the way back up to the top So you can kind of visualize how your code got to that point in time where it failed You can kind of reverse engineer what might have happened to get you there Now if you're dealing with a bug on a website, and it's on the front end It would be very helpful to grab a screenshot As well as note the browser and operating system that might be used in case it is a front-end JavaScript bug Now I work for an agency we develop Sites and applications for other businesses and sometimes our clients aren't the most tech savvy But all of them know how to take a screenshot Now you might get that fax to you you might get it pasted in a Word document But you're still gonna get that screenshot and screenshots are really nice because they're gonna show you what URL you're on What was inputted into the form what was going on at the time who they were logged who they were logged in as If they take a screenshot of their full desktop You can see the time and the date you can use that to cross-reference with your logs to see what's going on So screenshots are really powerful tools And if you don't get the date and time in that screenshot Maybe you're going through the site and you encounter the issue just shot down the date and time because again You can go and cross-reference the server logs watch dog logs And kind of see what was happening at that time were there any interesting errors or exceptions or unexpected events happening Basically, we want to define the symptoms of this problem and collect as much data as we can about it Once we've gathered this information now we can go and try to replicate the issue Try and figure out what steps do we need to go through to make this issue pop up When we do this we want to be able to replicate the issue with 100 percent certainty Let's say the issue is that Form submissions are broken Of course, hopefully you'll have more information about what form it is the error message you're seeing But when you want to replicate the issue you're trying to get that broken form behavior to occur Now if you can only get it to happen Every other time 50% of the time. How do you know later on when you fix it? Or you think you fixed it? How do you know that you're not just hitting that 50% of the time when it works? Not the 50% of the time when it's broken. So it's really important to try and replicate the issue with 100% certainty Now there are some cases You might be dealing with you know Competing locks or race conditions in your database or these crazy one-off issues that you've never experienced before That are really hard to replicate with 100% certainty and that's fine But in almost every other case you can replicate it with 100% certainty So try to do that try and figure out. What are the steps to make this issue happen? If you like to do automated testing go ahead and write an automated test that makes the issue occur because after you fix it You can run that test again and see that it passes and if not you can just write out a bunch of manual steps That's fine, too So once you have the information about the issue and you're able to replicate it with 100% certainty Now you can go ahead and try to identify the culprit go into the code and try and figure out What is the bug? Where is it happening? Why is it happening? When you do this you want to be very methodical You want to kind of have a hypothesis in your mind. Okay. I think it might be over here. Let me look over here Try not to make any assumptions without validating them first. So if you're dealing with a broken form Validate that the form data is being sent from the browser to the server Validate that is tricking the correct route validate that, you know, it's passing validation Don't just assume. Oh, I submitted the form is definitely going to this controller. No double-check that make sure that that is the case And once you find that culprit make sure you take time to understand the bug Understand why it was behaving the way it was behaving. Why did you see that behavior on the website? Why did it cause that exception to be thrown? Really try to understand the fundamental nature of that issue because that's what's going to build your experience in your intuition So once you found that issue now you can go ahead and fix it and retest So once the issue is fixed attempt to replicate the issue again Hopefully the issue will not occur and because you're 100% certain that those are the steps run through the steps No issue. Awesome. You've solved the issue Now when you try to implement that issue be sure to avoid that xy problem try to avoid Workarounds whether or not they're permanent workarounds temporary workarounds Workarounds are not ideal workarounds are going to add technical debt They might introduce other issues like in my story They might not even get replaced with a true solution later in my case again I work for an agency clients are paying me money to build their websites and to fix their issues They don't want to pay me twice to fix an issue that I already fixed So if I want to fix it the right way, I need to fix it the right way the first time They're not going to want me to go back later on and fix it again on their time So once we fix the issue and we've retested the final step is to mitigate future occurrences of this issue If you're doing automated testing add an automated test Maybe you wrote this earlier when you're replicating on the issue in step two Maybe you don't do TDD and you like to Create regression tests later on that's fine too add a test to your test suite The nice thing about tests is they take a few minutes to write and they're instant to run in the future So you'll know immediately if that issue ever comes back What mitigating future occurrences it goes beyond just preventing it from happening again in your code base It also means sharing your knowledge Let's say that the issue you were dealing with is a common misconfiguration issue Maybe something was poorly documented on the Drupal.org website or in your code base Document it say hey watch out for this make sure you do that instead If you learn something cool while you were debugging it Maybe you learned the internals of the form API write a blog post on it share your knowledge with other people if You asked a question on stack overflow and you didn't get an answer But you figured it out anyway go leave a comment on your post saying never mind. I figured it out. Here's the solution I can't tell you the number of times. I've googled a problem found a stack overflow post and the one answer is the person Who submitted it and they just had never mind figured it out Okay, what did you figure out? What did you do like you've spent this time to debug the issue you figured it out share that knowledge with others help them to avoid the issue or at least fix the issue quickly and Then if the issue happens to be in Drupal core third-party library or someone else's code send a patch upstream Say hey, I encountered this issue. What happened when I did XYZ? Here's the solution for it Maybe you know, you don't know the exact solution. Maybe it's a language. You're unfamiliar with I maybe you weren't able to fix it But you still kind of traced it back to that module or that package just leave an issue report with them say hey I'm running into this issue every time I do X Y and Z. Can you please help me? Anything you can do to help others mitigate future occurrences That's going to build, you know good karma if everyone if every developer is doing this That's going to help you out as well. So do what you can to kind of give back and help others So to recap this five-step process first, we were gathering information about the issue Get as much information as we can Then we try to replicate the issue with 100% certainty From there we go into the code identify the culprit Fix it retest it and do what we can to prevent future occurrences I think that adopting a five-step process like this has really good long-term results. I know it did for me For me it helped me gain experience again Experience with the code that I'm working with the frameworks I'm using the packages I'm using and the tools that I'm using to debug with It also helps me to learn how the system works because I'm diving deep into this code and I'm being persistent And I'm debugging and trying to figure out what's going on So the more time I spend debugging something like symphonies HTTP kernel the more familiar I'm going to become with that the more expertise I'm going to develop and This is also wanting to build our intuition. We're going to build this mental library of heuristics So the next time we see that connection timed out error or corrupt PDF We know what we wanted to go check first. It's gonna help us to bug much more effectively and Then lastly, I think having a five-step process like this is going to boost your confidence Because this process can be used in any language any framework any CMS The steps are the same even if the tools are a little different the steps are the same And if you're confident that you can go through this process That'll help you to work on those unfamiliar systems and in those unfamiliar languages So we talked about the five-step process What I'd like to do now is to kind of talk about some of the tools and techniques that I use during that process Some of the tools that I use to debug the issues some specific techniques. We're dealing with certain problems Now I am of the opinion that there are two essential tools that every developer should have Especially someone who's new to programming. I think they should have a good IDE and a good interactive debugger Now some of us more senior developers might say I don't need an IDE I'm perfectly happy with them or sublime text or nano and if that's you then more power to you that's awesome But newer developers. I think having an IDE will help them avoid simple issues Like syntax errors, it'll help them navigate the code base quickly Ideas have a lot of really good features to help you debug problems efficiently and effectively and in some cases to avoid those problems from happening in the first place a Good IDE will also be integrated with an interactive debugger like X debug if you've never used X debug before I highly recommend you try it out. You can basically pause live code execution You can tell a PHP. Hey, when you're doing when you're handling this request stop on this line I want to see what's going on at this point in time So you can stop on that line and then you can step through execution and say, okay, let's run the next line Let's see what happens. Let's run the line after that to what happens Let's go into that function and see what that function is doing And as we're doing that we can check out the different variables in that scope We can see how our code is modifying those variables and how the state of our application is changing And of course we can explore the call stack We can see how our code got to the current point in execution who called us. Who are we calling next? So an interactive debugger is extremely helpful for debugging issues Now in addition to these two tools there are several techniques that can be very useful When debugging certain types of issues, so I like to go through each of these in a little detail here The first technique is to trace backwards If we're ever faced with a situation where we know where the error is being thrown We might want to trace backwards to figure out how we got to that point in time Why is that piece of code running that's failing? So we want to fire up our debugger, set a breakpoint somewhere Run the code and see what's going on