 My name is Patrick Alar, the EE's mute, and you can follow me on Twitter or get my email address. About me and my accent, I'm coming from Belgium, so not far from France, same country than Greece or other PHP internals. I'm a PHP internals, I sometimes contribute to PHP, I'm also a RFC voter and I am the author of an extension called APM, so if you want to have more information that you usually have in an error-logging mechanism and something that is native to PHP, you just install APM and you can have your issues in a database or with factories, with additional information, get parameters and so on and so on. So, I also have a Drupal confession here, is that I have never used Drupal, but that's not the worst thing, I'm a core developer of another CMS. So, you can say that we have much more problems, so we need more debugging techniques, so feel free to do so. But debugging what? There are many things that can go wrong or you want to understand your system. It can be at the level of PHP itself, it can be at the level of your system, so outside the boundaries of the PHP process, it can also be related to networking. So those are the three categories that I'm going to cover. But before we start, there is one technique which does not fall into those categories, and for me it's the most powerful technique, I'm using it almost every week, and this is the rubber duck debugging. So I see many, many yes here, I think it's much more popular in the United States than in Europe, but who is used to rubber duck debugging? A few, I explained what the rubber duck debugging is, it's that for example you have an issue, a problem, and you decide to bring it on Stack Overflow, just an example. You write down your question and then you realize that the way you write it doesn't make any sense. You write your hands, who knows that feeling, oh I'm not alone, cool. Then you have to rethink exactly what information you need to provide to have an accurate answer, and sometimes you realize that you are just approaching the problem from the wrong perspective, and sometimes you start again from scratch and you find your answer. What happens here is that you find the solution yourself just by providing more information and you really try to figure out what the problem is to someone who has no clue about your context. So you have to give everything. So the rubber duck, I always have some cool rubber ducks with me. I have also a super rubber duck. So last week I had a good example of a rubber duck debugging. So no, I'm not crazy. I'm going to talk to my duck but it is how it works. So I had a problem. I put a die in my code, the die entered the code, but still I don't have the information I require in my database. So I reviewed the code and I tried to put some breakpoints and so on, but still no information in my database. So I took my first one. I took my ninja duck and I talked with him. There are many of them, big ones. So I think my logic is correct. Everything is good but nothing is in the database. Are you sure you know the stupid rubber duck question? Are you sure you do the... Yeah, I'm sure. Are you sure you're visualized the data of the correct database? Shit. And that's what it is. The stupid questions sometimes answer your question. And that's why a stupid rubber duck was more powerful than the breakpoint I had with the X-debug on the remote system. That's it. Apparently it works or so with real ducks. I never tried. Dead ducks too. You can't try at home. We have plenty of ducks. Unfortunately, ducks cannot solve everything yet. I like them. And because some of you don't know the rubber duck debugging, I'll offer you some rubber ducks. Feel free after the show. The first debugging techniques, which is the quick and dirty, it's not an advanced one, but it's one that is really effective. It's using dye. Do I enter this piece of code? I say code. And if you put a dye and you see that there is nothing that happens differently, probably you're not going to... into there. Often I use a burden, just to display the content of a variable. Often it's sufficient to have that to debug your problem. If this is a bit too complex, and I repeat myself, I do dye and verdance, and I change the content of dye, of verdant, then I'm heading in the wrong approach. Then I need a real PHP debugger tool, and this one, the most famous one, is X-debug. It's an extension that is separate from the gun, and it's written by direct presence. Thanks. Who is not familiar with X-debug? No hands. Cool. Because I'm not going to see X-debug. To me it's like regular PHP debugging, and I wanted to focus on something else. But if you have a question about X-debug, how it works, or want a demonstration, feel free to ask after this talk. Who is using PHP DBG? No hands. Do you know that PHP DBG comes by default, since PHP 5.6, and that most certainly, unless you have an old version of PHP, if you just write PHP DBG on the command line, you're going to have it. Look after that. This new color is native with PHP 5.6, but it does not replace X-debug, the one. For use information, it lets you breakpoint, it lets you do step-by-step debugging, but it's mostly a command line tool, which is quite similar to the C equivalent DBG. I can show you how PHP DBG works with slides, but honestly, I prefer doing it in live. So I'm going to do this, so that I can still hear my voice. PHP DBG, on the command line, I have some files here, like a test file, which contains some greetings. If I want to run it, I'm just writing PHP DBG and then the file. If I just do this, it loads PHP DBG, but it doesn't run the content of my test script for now. What I can do is testing the content. If you want to know more about the possible commands, just type help, and it shows you an explanation of all keywords. So I'm going to list it there again. I want to breakpoint on something. I can breakpoint on the line number, on the name of a function, on the class method, and so on. Just use break, for example, to stop on, let's say, line four. Four. Yeah. If I run it, I see that I am currently on line four, where hello is going to be assigned to the variable hello. If I write print, oh, this is a big one, I am going to see the operands behind the current code. So what are operands? When you write HP code, it's text. Before it gets executed, it will be first transformed to operands, bytecode. And then this bytecode, if you have bytecode caching mechanism, like opcode or APC, it's not going to interpret the text every time. It's going to run the various opcodes. This is a bit cryptic on that one, but when you see assign, it's just the assign operator. Rope up, rope at, rope ends. This is for string manipulation, like concatenating. Thanks a lot. So list the 10 I mentioned here, just the number of lines I want to display. In this case, I can evaluate things. For example, I want to know what undefined variables, so I have the true key. I'm really inside the core of PHP itself. I think I just have to step one more line. I see that it creates an assign statement, so he assigned hello to the variable hello. And now if I evaluate words, of course I have to evaluate hello. Yeah, I correctly have a hello. So this handy. I'm going to show you some other examples. I have a loop, an infinite loop. It's going to write a dot and so on. So with that one, I can also interfere with the content of the engine. Oh, no, list 10. I'm going to break on 6 if I run again. That doesn't work really well. So I want to break on, let's say, 5. So if I do next, next, next, I'm going to jump between the two operations that are made here. It's the echo, that's one operation. Condition, that's the other condition. I can do this forever and ever. It's not going to stop. But for example, I can say that I can evaluate some codes and now doing next and now the script ended. Because I really interfered with the content of my variable, which is interesting to see what happens in other condition. It can also display you the various uploads of a little script. So who of you is aware that single code strings are faster than double code ones? A few hands. Who is aware that it really doesn't matter? Okay, so it's supposed to be a myth and it is really a myth. Single code has never been faster, even if you have used the dollar sign into it, a string interpolation. For example, if I have a little script, single code of PHP, or one that is double code of PHP. So I have a concatenation of two strings, one with single codes, the other with double codes. And if I want to ask DBG to print me the content of, let's say, the first one, single codes. Here you can see that before while understanding the various operations, instead of having a concatenation of hello and words every time, here it's going to assign to the variable greetings hello world in one block. Here I see double codes, that is just the output format of PHP DBG. If I'm asking the same thing with double codes, I'm going to have exactly the same output. So the operands that will be stored in memory and being run again and again by your output cache is going to be exactly the same operands here. So if it really matters, try to not use single string just for performance because you have less feature with it than with just regular double strings. Is this clear? Yes. Cool. PHP DBG can also be used to debug something. If you want to emulate a full request with some underscore, dollar underscore get, dollar underscore post content and so on, you can create a script file to emulate the environment and pass it to PHP DBG. In the slides, I wrote some examples so that you can later, after this talk, find how I did in the live team. So I'm going to pass this. The other tool, this one is not native, it's PHP trace. There is a lot of tracing tools that ends with trace. We are going to see a few others. And PHP trace, it's a very low overhead tracing tool. With it, you can connect to a live PHP process. And as soon as you are not entering the debugging mode of PHP trace, there is no overhead, really zero. So how it works, PHP trace, there are two tools. One common line tool that is going to connect to your PHP. But to PHP, to be able to talk with this common line tool, it requires a PHP extension, which is also PHP trace. Everything is combined in one project. So you have, like it's a bug, you need a client and some remote code. So if I'm going to create a loop that is just going to sleep a little bit, I will again do it in live. I can run some PHP code just on the common line. Here I'm maybe the blue, so it doesn't really matter. So here I created process with process ID that is put here. So it's another one, three, zero, four, eight. So every time I enter PHP trace with status, I get a status of the process with how many memory has been consumed so far, the big one. I see the arguments of the current score. For now I'm in a for loop. And I can see the back trace of the moment I did the PHP trace status. If I don't provide the status keyword, then I'm going to attach the process. And so for now the process there is zero overhead as soon as I press enter. Now then there is a little bit overhead added to the process because it dumps the information and I can see directly in live where this process is actually. As soon as I interact the PHP trace, the overhead that was generated because of it is gone. So you can safely use PHP trace in production. If you have something that is stored or in a kind of infinite loop, it's very hard to debug because most of the time you want to reproduce it. You don't know how. You already have the case but you don't know how to get there. So it's very interesting to have PHP trace in production and to be able to attach to a process. So the argument I'm passing here is the number corresponding to the process. In this case it would be, in your case, might be Apache, PHP, FPM and so on. System debugging. I'm done with PHP because sometimes you have to be outside the borders of PHP. We're familiar with S-Trace, a few hands. So S-Trace is tracing the system calls that are between a process, an application and the system. What does it mean? If I want to open a file, for example a NINI file or a YAML file, I want to read it. The system is going to use maybe open, the open function in PHP. But this one just cannot access. I want to read the disk at this place. This is an operation that is handled by the operating system. I ask the operating system, okay, give me an access file descriptor so that I can find this file. It's going to say, okay, you don't have access to this file or yes you have. Here is the file descriptor. And then I can do some processing. This communication is done with system calls. So an example of system calls is F-Open because the system calls and the function in PHP is very pretty similar. It's almost just a wrapper around the system calls. So again, live demo. What do I have here? ls.php. I don't know why that is. That's going to open. Maybe I was debating. So ls.php is just emulating ls. Okay, not really, really fell, but it does work. S-Race is quite verbose and you can just place it. Excuse me. You can just place it in front of the thing you are going to execute. You can just do ls. Just show this or I can use s-Race and then I do ls. Then I have my output. Here I have my output that is combined with the output of ls. Not really practical. Oh, sorry. At the end, can you read what's on the screen? Okay, perfect. Yeah, a little bit. I know. I just asked s-Race to put the output in another file. Thanks, Vi, for the colors. And here I can see all the different information between the PHP process. So it means even the internal code of PHP, including my code, everything is shown here. So most of the thing in the first place of the file is about loading the PHP executable itself. Even without providing anything, you are going to have some system codes. For example, the PHP process is going to read a PHP Enify. To read a PHP Enify is going to parse some directories until it finds the one you want to have. So the good thing is that I can use, for example, that example to find if something has been done on... Oh, no PHP.ini. Oh, yeah. Why nobody tells me that I was doing ls? Thank you. So here, I do it again, not with ls implementation, but with the PHP one. So I have some output. So here I have all the output of PHP. Yeah, it's much, much, much bigger. Might be quite verbose. But now, if I look at PHP.ini, I'm going to see where it tries to... So here we have an open. It tries to find PHP-cli.ini at this path. E no end is the code return when something does not exist. It means no entity. Then next one, next one, and so on. And here, he realized he found a PHP.ini file and it returns me three. Three is just a file descriptor. So if you output the result of an fopen, the number you usually see is a resource that corresponds to this one. Of course, this is extremely verbose. It contains all kinds of system calls, file system calls, networking system calls, memory mapping system calls. If you want to filter, you can just provide the category you want to filter on. So here I want to specify the file category. Now I only have the system calls related to the file category. So it means open, access, sxv, and things like that. Less output. You can also remotely attach to a process with minus p and then the process id in a very similar way than with PHP trace or the other tracing tool. If you want to have a summary, system calls by system calls, you can pass minus c and you have a summary of everything, count numbers and so on. For libraries, it's quite similar to L trace, sorry, it's quite similar to S trace, but for libraries. Libraries is, for example, PHP is not done with external C libraries. If I look where is my PHP, U.S. Robin PHP, I do LDD, U.S. Robin PHP and I'm going to have some output. What does this mean? PHP didn't reinvent all everything, so if you wanted to use and reuse existing libraries, like, for example, every time you use the girl with girl in it, it's going to use under the hood this library. And if I want to look at all the communication, I can do something very similar, but beware. So here I have an empty PHP file. I just want to see the course to the Lipsy library by doing PHP empty. And now I'm taking a coffee. At some point it ends. Try not to run this on a heavy, heavy, heavy project. It's going to be run for a long, long time. Here I can see really what are the specific functions of every libraries that are used by PHP and this recursively. So if you call PHP to resolve, like, for example, get host by name, this is normally something that is handled by the libresolve.so file. And this one also needs to open a file like etcost, for example, and you have some chaining between the various libraries. You can dig into them and try to visualize all of them. It's pretty handy, but it's very, very verbose. So most of the time using this tool is whenever you know you have something wrong. You have an HTTP proxy in between while using CURL and you really want to figure out what's going on at the library level. That's the tool you need. You can also make it less verbose by providing the exact library you want to filter. And you can use it without this country, but we'd like. iNotify. It's not a debugging tool, but who knows what iNotify is? One hand. Interesting. So iNotify is a Linux feature that is pretty similar to the event system of Symphony or it's like an observer pattern on files. I want to know whenever you touch this directory or this file or you open it, you access it, whatever. I want to be notified when you do that. And iNotify has nothing to do with PHP or C or whatever. It's really a Linux feature and I can demonstrate it like I'm inside the same directory here and here. iNotify is a kernel feature, but if you want to have some command line possibilities, you have to install an additional package. Most of the time the name is iNotifyTools or something very, very similar. The two binaries I'm using here is iNotifyWait and iNotifyWatch. I'm just going to see how I can use it. Here I want to be notified of any change recursively made to this current directory. The watches are in progress. Here I just want to do, for example, ls. And now I can see on the other part that the Linux kernel system sends a signal to iNotifyWait to prevent, okay, you had an open on the directory, you had an access on the directory, then you had a close without right, a close and is there and so on. So even with the very little info, for example, if I do a cattest.php, here you saw some output and I didn't press the enter. Why you can see something is because the tab completion internally use something like ls to know how we can complete and because of that you also try that command. So this is quite cool and it's not only a debugging tool. If you want, for example, to program with iNotify in mind, for example, you have an upload directory and every time there is something that needs to be uploaded to this directory, for example, an image, you want to remove some geotags to it and so on, you are not forced to do it into your PHP process. You can ask some tools to be noted arrive by subscribing to that kind of event and then you can easily crop things, put a watermark and whatever you want on the image. It's not a heavy implementation because it's not actively... Okay, is this fire touch? Is this directory touch? No, that's something really, really small. It's whenever a system calls like open to a directory is performed and there is actually a watch, then the kernel is going to send a notification. So this is a very efficient notification mechanism. So again, I haven't mentioned iNotifyWatch I'm running a bit out of time. So iNotifyWatch, it's exactly like iNotifyWatch, the thing is that it doesn't produce an output until you hit control C and then you have a summary of every operations on all directories and all files. That's very handy. So why debugging tool? Sometimes I just want to know which files is touching something or when it is touched. So for that purpose, with S-rays, it's very handy. My SQL proxy, same question. We used it on hand, not so popular. My SQL proxy is a proxy. Yeah, you guessed that. And the proxy means it sits in between. So what can be used? What's the use of my SQL proxy? You can do whatever you want it to do because you can attach to it a little lower script. You have plenty of existing lower script for my SQL proxy. For example, you can say you have an intermediate proxy server. As soon as there is a request for it, I just want to log it in the file and then continue. Every select operation using that table, I want the same select with an explain in front of it. And the output of that, I want it to be in a file. So you can programmatically create a proxy between your database and your program so that you can interfere with how it lives. Another example is if you want to, for example, query, want a log of everything that is not using an index or log every queries that took more than one millisecond and things like that. I'm going to show you. You don't see the script? No? So my SQL proxy, what I have to do is calling my SQL proxy with a script. Here it's a very simple debug.lua script I created myself. Those scripts are online on my GitHub account so you can take a look there after. So here I just enter it. And I'm going to change in one application the port. I'm not going to connect. That's the configuration of my application. I'm not going to connect directly to the usual port, but on 4040. 4040 is the one that is used by default by my SQL proxy. I'm taking my application with me. Actually I'm not going to show you this. You don't want to cry. But you are going to cry anyway because you see the output of the SQL generated. Here I just refresh my screen and there isn't a lot of queries performed to this. The cool thing is that I can directly visualize what's going on because this is just a script that outputs things. If I have my cursor, I want my cursor, yes, and I'm using something like grep and I want some from-users space dot star home. No, sorry. This way I'm going to print only some records. I just refresh the screen in my page and I found some queries to the users table where users ID equals something and it didn't return any results. In my script I wanted to write known in red because doing a query and you will receive no results. It might be WS, so it might be wrong. That's an example of Clusage. You can just combine it with grep. In the repository I have online which is going to be here. I provide various different scripts. For example, if you want to emulate a database that is very slow you can easily create a script that just adds a sleep into it. That's a bit nasty because sometimes if you have one page that is doing a lot of queries you may not notice because still your environment is very fast but if you do a micro sleep of just a few milliseconds you can notice because in the end you are going to see one screen that is going to take five seconds instead of five milliseconds. There is another script that, for example, just sleep but for the same amount of time then the query that the time spent in the query. It's just about double the times and you can add yourself a factor. If you want to emulate a database that is five times slower or anything, you can do that. If you have multiple databases all selects need to be to the local database while all updates, deletes and insert statements need to be to a master database. No need to configure two connections one for read and one for modifications. You can do that with my MySQL proxy script. Nethawks, that's for networking. I'm sure you know the top commands. I haven't wiffy that works well enough to show you but it's like top but for the networks. It requires root access. No devices to monitor. It means my wifi is even not open so I cannot even enter the command. That's the first time I encountered this. Nethawks is going to display you per process the bandwidth usage so if on server you realize that you are a little bit out of bandwidth and you want to know from which process it comes from which one is your bottleneck with Nethawks you can just see how many megabytes one process is going to consume. You can do that in front of Composer. For example, it's quite interesting. You see, it's going high. No more things to mention. Wireshark, this is TCP dump on steroids. You are using it? Yes, many hands. I'm going to finish with a little presentation of Wireshark. Wireshark is a tool that can replace any charl proxy or tools like that. It's a general purpose debugging tool for network. It can analyze TCP, UDP, whatever providing some kind of filtering mechanism. So if I want to, for example, TCP port, I'm going to TCP port HTTP. I guess that would be a good example. And here if I hit my, if I have something here. Yes. So normally here I see all the packets. Those are TCP packets. So it's not very, very readable that way. But those are at certain moment regrouped. For example, from here to here, it's going to be the TCP packets that forms this HTTP message. So if I want to filter and see only HTTP message here, I do the filter here. I see only HTTP message. And now I can see the various information. I have plenty of them. Like here I can see the content length, cookies one is one. Like you have usually in Firefox, Google Chrome and tools like that. You also, you have a net console. But sometimes your application is in a client server mode. Like a RESTful application and things like that. Or you are doing this remotely. That's difficult to use tools like Google Chrome or Firefox to handle the traffic. And most of the time this is for HTTP. So as soon as you want to debug the same thing, but at the MySQL level, see what's going on between your client, your MySQL client and your MySQL server. You can use the same tool, whatever the protocol is used. Wireshark lets you also create these sectors. So if you are having your very own implementation of HTTP or REST and you want to create your own inspector to see the traffic, you can quite easily create such a sector for Wireshark. It's just a bit of C++ knowledge, I think. And that's going to be everything. Do you have questions? Because I've presented a lot of things, so I'm going to add a lot of questions. Fire. Yeah, sure. With tools like you mentioned, is that actually usable for larger applications? For instance, you mentioned easy publish. Like if I have an error that may be coming through in twig and I'm trying to debug it and it's in bundled and it goes through twig processing and so on, I can't just start at the kernel file and then just walk through all the code that's executed, you know. Is there a usable way to set the approach whenever breakpoint be down in the code when it starts from the timeline when it wants something in the work process? Yeah, very good question. I use PHPDBG a little bit like I use DBG. It means with plenty of macros. PHPDBG comes with an initialization script so that if you want to create your own alias so that you don't want to repeat yourself between every request or you want to easily bootstrap a request with some data into it, you can easily reuse an environment that is already set up for you and you can also easily share the macros on your project because sometimes there are typical points where you want to stop and you can share the PHPDBG initialization script in your git so that the rest of the team can also reuse the macros. PHPDBG is so easier than it's about to use remotely when you have different ops and you don't want to set up tunneling things like that. So in terms of size, I would say it's interesting but it doesn't replace it with a bug. It's not meant to. It's a complementary tool and I don't think the size really does matter for PHPDBG. DBG is very similar and you can use it on, for example, Linux kernel to debug something. So if you have any more macros and you have your habits, it's quite efficient. Did I answer the question? Yeah. Somehow. Yes. Can you please use the mic? It's recorded. Thanks for the talk. Most of these programs that you've shown us, are they run on PHP 5.6 or higher? Or are you going to use them? So the tools that is specific to PHP here, I mentioned, is PHPDBG and PHP Trace. Both exist for PHP higher and lower than PHP 5.6. PHP Trace, I think it existed since PHP 5.4. So hopefully you are not using PHP 5.4. No. Yeah, okay. If you... And PHPDBG existed before PHP 5.6 as well. It's just that it is now natively included in the core of PHP. So... And all the other tools are not related in any way to PHP. So you can use all of them. Other questions? No? Okay. Ah, yes, one more. Sorry for the oldest. Do you have any recommendations on resources for that kind of wire shark a little bit better? I found something a little bit difficult to get into. It's just sort of like decrypting TLS and something. Yes. So wire shark, if you want to... If you want to debug HTTPS traffic, you can use that. So what you absolutely need to have is the private key of your server. You need to set your wire shark to know this private key. And then you are able to debug your HTTPS traffic. It depends on your setup, but sometimes you have a load balancer that does the HTTPS for you, and then you have internally HTTP. So it makes things simpler, but it's not always the case. So thanks for the remark. Last question? No? If you have any questions regarding those tools or anything else, easy publish, or PHP internals or whatever, just feel free to catch me. Thank you very much.