 Our next speaker will be Ike DeWolder. He's one of the regulars in the PHP Belgium scene. He's also an organizer of PHP in Belgium. He's going to talk about something I think we all have and that's namely profiling PHP applications. So Ike, go ahead. So yeah, profiling PHP applications. First, can you hear me in the back? Yes. Yes, okay. That's good. So first, who of you like slow applications? Yeah, you can leave now. We need chairs. Is it only about faster execution time or would we like to use less resources or is it only as much request as we can get? And usually what happens is we start profiling when we see this. So everything is on fire. Your client is on the phone. Yeah, my site is down and everything is slow and it breaks. And yeah, what's going on? You don't know. You have no clue. So that's why we are going to profile our application. But first, hi, I'm Ike. He said it already. So short introduction. I'm on Twitter. You can send me things on Twitter, whatever. I do some development, a little bit of operations at Studio Emma. We do Magento and PIMCore. If you want to know something about it, find me afterwards. That's it. So first, profiling. Profiling is a process where we find the runtime characteristics and do the performance analysis for optimization of our application. Is that clear? So what information do we get from this profile? Where is something happening? Where is something slow? Where is something using a lot of memory? Such things. What is, well, we want to know what is causing this, but what is using it? That's the first thing we know. We also learn how much. That's already a very interesting part of it. But then you have that question, why? My application is fast. Yeah, you see, if I click it, it's fast. But do you have substantial information to confirm that your application is fast? And what is fast for you? So define fast. Is two seconds fast for you then? Yeah, OK, fine. Don't profile. But can you handle a burst of traffic? So your marketing department has a great idea, and they don't tell you. They will send a great marketing campaign, and then it happens. Bam, everything down, and then you ask around. Yeah, did somebody do something? Oh, yeah, we sent a mail to all our customers. Yeah, thanks. So optimizing your application has benefits in all areas you can imagine. There you get this. Yeah, I don't need a profiler. I know what's slow. Are you sure? You're kind of shooting in the dark. If you are, yeah, you probably can make an educated guess where something is slow. But if you don't have the numbers for it, it's still a guess. So the points of interest we are going to look at while we are profiling are execution time. So this is the overall thing. Yeah, we get a response in two seconds, or maybe we like one second better. But we also get the CPU time spent on your application and the memory usage. Also interesting to have is how much disk I.O. does my application cause, because disk could really help you if you get a burst or something. What we also have is network time. Do we lose a lot of time asking using external APIs or same with external delays? Like is the response from your database slow? Or did you just kill your Redis in some magic way that is suddenly super slow? Or other databases you are trying to use? So now, how can we profile our application with PHP? Who would do it with PHP? Just something in PHP magically and we'll get something. No one? Yeah, maybe not. So you can get some information. You get the time. And if you do request, you can even get it from your server super global. But yeah, you get actually nothing because you get it already from another way. So first, XDBug. Who of you knows that you can profile with XDBug? Oh, that's already quite a lot. So XDBug is an extension. You know it for PHP. Usually we use it for debugging, but it also provides us with profiling. It needs a little bit of configuration to make it allow you to profile. And you need an external tool to analyze what it creates. So it creates a cache-grind file and then you can visualize it with some external tool. So what I usually use is enable the profiler, write it somewhere, and cache-grind out percentage U. That's just a timestamp. You could be smarter about this and get more information. You can find it in the documentation of XDBug. And once you get those cache-grind files, you can open it in a tool like Qcache-grind or Kcache-grind or whatever, blah, blah, blah, cache-grind tool there is. I think there is a web-based one also. Then you get some information. But it's not super easy to understand what's going on. You can get some visualization, but there is not really something showing up as being very slow. And there is one larger drawback with these tools that they have a maximum. There is a limit on the number of function calls it will show you. So usually you can get it up to 500 function calls. But this comes from Magento. This is a very complex PHP application which has a lot more than 500 function calls. So you will probably not see all the information you want. Probably what you need, but it's not very easy. So it's hard. Also, XDBug is not really a profiler, but it's a debugger with profiling capabilities. So it also introduces kind of a large overhead. It became better with newer versions, but still the result might be a bit off. Then I guess the people who knew XDBug also knew XHPROF. But maybe there are other people who also know that XHPROF is out there. So XHPROF, well, these days with PHP 7, we don't no longer use it as XHPROF itself. But it continued in the Tideways PHP extension. There is another open source extension that has similar capabilities. But personally, I use the Tideways extension. It's also a little bit of configuration needed. And you use, well, in my personal preference, XHUE, which is an external tool to visualize the profiles. So it's very simple. The only thing that's really important here is that you do the Tideways auto-prepand library on zero, because that's for the Tideways SAS platform. You also include some prepands, where you actually initialize your profiling. And it's not all there. But eventually, if you get in the shutdown function, you collect your information that it gathered from your application and write it somewhere. That was the wrong way. So then you have XHUE, which is empty in the beginning. And once what I also forgot with Exdebug is that everything you do with PHP is profiled. So you do not have to do something extra to have a command line script profiled. Because if it's on, it's on. If you do command line, if you do request, whatever, you enable it, it's always on. That's, for me, it's very interesting. So then you get some runs in it. You get an overview. And in the details, you already get some information. This slide contains something that might be interesting, because it already has an indication of a problem. So at the end of the presentation, if I didn't go deeper on to Composer, autoload, include file, ask me about it. What's the problem? You get your function information annual, so you can get a call graph, which might be overwhelming. But it's very interesting. So for me, the important thing here is that it's still a local setup, so I don't need any external services. And it profiles everything. It doesn't look too fancy, but it has most, well, all the features you want to get some information out of it. You also have Tideways itself, which is a SAS that you can also configure. So it's also the same thing. You get an extension. You configure it. It needs a daemon on your PHP application to send the information to the platform. The interesting thing with Tideways is that it has a 30-day trial that offers you all its features. So you really can evaluate what it can do. It looks rather nice. It also has APM functionality that is out of the scope of this presentation. And if you want to profile something, you use an extension, so the profiling is on demand of you. You say, I encountered some sort of problem on this page. I now want to get a profile out of it. So you say, take profile, and then Tideways will reload this page and give you a profile in their dashboard. It gives you some information about the function calls, when, what happened, and so on and so on. So it's more than a profiler. It's fairly easy to use. But yeah, it's trial to paying. This really depends on if you already have an APM in place, and you only need to profile locally, or you're really new into this, and you can benefit from a platform that has everything. So that's the kind of a choice you have to make. And then we also have Blackfire, which is very similar in setup to Tideways. So it's again the same. It's an extension. It's similar. It uses a daemon to get the communication done between your application and their platform. The biggest drawback for me is that it's a freemium model where their free features are not really useful to evaluate what the power of the platform is. So you actually should start with the paying version where you see everything and then decide if you want to continue with it. So it's exactly the same practically, like with Tideways, you click a button, and then it profiles a page, done. You get some information. I only have screenshots from the free version. So it's very easy to use, but the drawback is in the freemium, you only have the wall time and the memory usage. If you want the CPU time, if you want to see IO, if you want to see network, if you want to see all the other things, you really need to start paying from day one. That's what you choose. That's for me the biggest drawback. It does not offer you all the features to evaluate if you want to continue with this platform. My biggest suggestion here is pick one and learn it. Once you learn the one you have picked, you probably will encounter things you don't like about it. So you don't find your way properly in it, or it doesn't offer sufficient information on something specific you would like to know, and maybe another platform offers it. Then it's time to start looking around for, start shopping, you could say. But first, you should dive into one to really learn its quirks and how it's going to behave on your applications. It's especially interesting if you already have in-depth knowledge about your application, you will quickly learn what is missing in the profile or that you are using. To summarize the tools, it's like Xdebug, Xhprof, our local setups, Tideways, and Blackfire are SaaS solutions. So if you really like the SaaS solutions, go for one. If you want to keep it local, use Xdebug or Xhprof. But now, we still need some idea how we are going to approach the profiling itself. So what are we going to look for? What are we looking for first? We look for slow things. For example, the Contact page on the Magento website, if you configure it default, it's probably the slowest page on your site. Weird, isn't it? Also, what you really want to optimize for is popular things. So if you're in e-commerce, probably your catalog pages will be hit a lot more than a product page. So if you really want to profile to gain, well, to be more future proof, you are going to watch at the popular thing so you can preemptively optimize them before someone trashes your site. And application initialization. That's the one we saw at the earlier slide, where composer autoload include file is really slow. If there is only one takeaway from this presentation, remember if you see this composer autoload include file that is really slow, just think opcache memory is too small. This is not obvious. But yeah, we searched for it a few hours to find out that actually what happened was, oh, composer wants to add a new class. So it includes the file. But the opcache memory is too small. So yeah, the one that was already in, well, many others were in. But the oldest ones were pushed out. But they were used again a little bit later. So it continuously accesses the disk to get the information instead of its class definitions coming from opcache. This was a not obvious issue. So and then you go and analyze the profile. First, follow the hot path. So usually it's marked as the more red, the slower it is. So usually if you see the red fading away, that's kind of the point you're going to start to watch. And yeah, what's happening here? Does it do a lot of MySQL calls? Or does this method does something really weird? Like I saw an example where, I don't remember where I saw it, but it was another presentation where it was like, yeah, we have an HTTP client and it does something really weird. And far, far away it did something like, I calculate pi to 20,000 numbers behind the comma. Yeah, if you want to do that, but yeah, that's weird. So it can be weird code. So fix that. Try again if it's better. You're good. So now a little demo from something we did in real life. First I'm going to start with a more simplified example. Is this readable or do I have to zoom it a bit? Zoom, yeah. It's good? Yeah. So there is some background to it. Yeah, that's the one. So the story here is we had to create an application where they said, well, we're not going to use euros. We're going to let our users buy some sort of coins. And not crypto. Just you give 60 euros and you get one thingy. And our products will all be based on, yeah, let's say we take a base value of 60 euros and everything will be based on that. So usually something we will sell will be one, two, or three coins. Yeah, OK. So something was built. And it was like, yeah, someone wants to buy coins. So we started with, yeah, the coin value is like, well, let's take 100 euros. And yeah, we want to be generous. And we're going to give them to other people. So we'll buy 10. Oh, that's too fast. Yeah, sorry. So no, we want to buy in euros. So we want to be generous. So we want to have 10 coins to distribute to other people. And that's pretty OK. So initially, if we're going to look at this, so we get here one run and it takes one second. So we get into details. Keep in mind, this is a very oversimplified example to show you an easy part into profiling. So what I usually do is, yeah, I go into call graph and then it's always too big. And now there is just a cheat in it where the sleep is now taking most of the time. So this is the reference. It takes one second. So our code we are using is OK. At some point in time, there was another department in that company that said, no, no, no, we're not going to do that. But the whole idea of their website was based on the coins. So then they said, yeah, let's change that. We're going to use coins of one euro and go wild. So yeah, with the thousand, it's probably OK. So if we're going to look at it, it's not really that much. Doesn't really matter. And then they said, yeah, but we have a lot of our own personnel and we want to give them a budget of 2,000 euros or something for courses on our own platform. So I'm going to buy for 100,000 euros and then distribute them across the company. That's when we started to see weird things because the platform was completely not built to have 100,000 coins ordered at once. So it's becoming slow. So we know the reference is one second. Now we're at 13 seconds. To get the idea, we started looking at things in the profile. The Zoom is working against me. Yeah, that didn't go well. Yeah, that's fair. So what we come to see here is we follow the hot path. And yeah, it's probably too small now. So we follow the hot path. And instead of scrolling, I want you to Zoom. Thank you. Yes. And it's like, yeah, store coins. What the hell is this thing doing? It takes 10 seconds. Probably should make it better. So if we look at it, it's based on an actual, it did happen. So demo service, coin service. This is actually very readable. So what happened is there was the idea of, yeah, we go to the slow one. So we store things. And what happened is, yeah, just add one coin. And the table contains one coin per row. So we start to insert 100,000 coins and go wild in a foreach. And then do other stupid things like try to check how many unspent coins there are. So the combination of these two is really disastrous. So because we didn't originally create our system to support really small values, it escalated quickly. And here you can see you can easily follow this path to find out where things go wrong and start refactoring here. Because we do not actually want to insert 100,000 rows. We don't really want to do 100,000 queries. We really want to do one query to get all these things in. And it will be a lot faster. So this is a very obvious example. But we'll get to others later. First, we have some popular bottlenecks. So I already said that. Slow code, really weird algorithms, or spending too much time on things you don't really want to spend on. Bad SQL queries, what you also get is in the beginning, your site might be very fast without indexes on your tables. Eventually, you might see slowdown. So use your indexes, but don't overuse them because that might bring you to other problems. What we also see a lot is disk IO usually triggering your application to be slow. And usually it's a combination of something externally is doing too much disk IO on your server. Your PHP wants to do a lot of disk IO and suddenly things go bad. So what could happen is suddenly backup kicks in and reads a lot of files without IO limiting or whatever. And it causes your application to go slow. What we also already saw is usually we have an application and it has a cron somewhere. And the cron goes wild because someone did not tell you they were going to do some marketing campaign or update everything at once. And once you see swapping because you have memory issues on your server, you have to think about other things too. Also, if you want to be fast you need to keep in mind that not everyone wants to be fast. So what we also encountered in the past was like, yeah, we have an API. Do whatever you want with it. But they found it acceptable to have a response time of one second. But if you want to go fast, yeah, you have to do something about it. And then it happens. Everything goes down and it breaks. And there they are, the angry customers. So the first thing you have to do is don't panic if your site goes down. Just relax. It's not going to get any better if you're panicking too. So first, just throw in more CPUs if it's needed. Throw in more RAM and then see to block probably some malicious actors so you probably find some bad bot list and block them. And when things start to normalize, then try to find out what happened and try to find out how you're going to improve it. Then find the slow parts, start profiling and get on with it. So for your production environment, it might be interesting for everyone to have your monitoring have an application performance monitoring on it. What I find personally very interesting is to have ATOP on it. I don't know how many people know ATOP. But if you have it on your server, you can, when everything is back to normal, you can try to recreate what happened, see when it started to explode and get some really useful information to avoid these kind of issues in the future. And if you really need to, you can go and S-trace things if you really want to go deep. So I have one other demo where it's not exactly obvious that something is slow. This will be, so this is Magento, there are no products in it. So it's, yeah, it's empty, but it doesn't matter. It's going to show up that something is slow. I already said it, it's the contact page. So if you go to contact, it's going to be around one point something or two seconds. And if you go look at it, we see like the default was 380 milliseconds and the contact page did 2.7 seconds for a page with only a form. It's weird. But if we go into that, yeah, there is a lot of stuff going on here. This call graph will take a while because, and it's going to be intimidating. And it doesn't want to zoom again. And then it was gone. Oh, it's too small now. Yeah. So it's back. Yeah. But you, as you can see it, there is a lot of red everywhere. And yeah, where to start? You will need some inside knowledge of your application. You will eventually follow the hot part again. And we'll find out something peculiar. I think it's there somewhere. No. Yeah, let's not try to find it live. It doesn't help. But somewhere, if you get into this problem, you will find out somewhere that it renders your page in PHP again. That's kind of an expected behavior because you thought, yeah, well, I have all my caches on, so you don't have to re-render this page. It's somewhere in cache. If you dig a little bit further, you will find out it has a magento capcha in it, which is rendered server-side. Surprise. You cannot cache this page, so it slows down a lot. But for these kind of problems, you also already need a lot of application knowledge, well, inside knowledge about the application you are using to find out actually what is going on. So you will eventually recognize a problem. Oh, it renders the HTML, so this is not what I expected. So why does it do it? Eventually you find out it always renders a capcha. You don't want it to do, so you need to find another solution. So, in closing, find your slower things. For me, my preferred go-to solution is xhprof and xhui, because we use an APM in production, so we get some starting points from there, and then we only use profiling locally to really fix the problem. Also, don't trust profiles from your application, performance monitoring, for example, when your system is under stress. So if you have a high load on your server, that information will not be useful, will not be correct. Also, what you can take away from this is make your popular pages cashable. It will help you a lot. And if you can profile periodically in your development environment to, if you have some reference, you will know that your application slowed down while adding new features. For example, you had to do a bug fix, and it suddenly, one and a half seconds instead of one second, maybe you have to reconsider the fix. Are there questions? Yes, so the question was, have you ever tried New Relic? Yes, we use that as an APM in our production environments, but that was out of the scope of exactly profiling. So that's also the reason my preferred go-to solution for profiling to improve things is XHUE, because we get some pointers from New Relic where things are probably slow. And if we can reproduce this locally, we can get ahead. So you already get useful information from New Relic, so you can reduce your search time in the call graph where you already know from New Relic, okay, it's about my SQL queries. It's elastic search that's responding slow. Something else, so yeah, it's useful. You have other competitors of New Relic. I think Datadoc is one, and Tideways also offers you a minimal set of APM. So yeah, it's very useful to have it. The only drawback we've seen lately is that in really complex applications New Relic might have too much overhead. So we switch it on and off. If we expect a problem, we turn it back on. But if we want normal operation, we turn it off because we notice at some sites between 15% and 30% overhead, which is really big. Yeah, the more complex your application becomes, the bigger the penalty is of New Relic, even though they are actually a tool that you want to run in production, that you want to have in production to get this information. But yeah, we've lately seen it if your application is really big, it might become no longer really beneficial to have it. Yeah, for example. Other questions? Yes, so the question was, is there a possibility to get this in continuous integration? I summarized it correctly. Yes, it's possible. So that I didn't mention is like Blackfire has this, that you can make it profile during your continuous integration. So you really see the trend if your application is slowing down over time. So you can make it a part of your continuous integration. It might be very useful. Also, what I did not mention is there is a newer tool which is called PHP Spy, but I only tried it briefly and I'm not sure if it's really that easy to use for request-based profiling, but it works very well for command line tools to profile those and it gives you a nice fire graph which is very easy to read and to understand. But it's fairly new, so I have to dig into that deeper. More questions? No? Well, thank you and you can give me feedback on joinedin or on Twitter or whatever. Well? Yes, very good. Nice time. 43 minutes.