 Hopefully you can understand me. I've came over a flu on the flight over. I'm not there right now, and I was earlier there. So web application debug toolbars exist for various web frameworks. Solutions exist for frameworks input in Python, as well as Ruby, PHP, and pool. Now I have no idea where that idea first originated, but for Python at least the first example I know is the Django debug toolbar. Although I'm sure if we have some sort of thematic in here, he'll jump up and down and say, no, no, you're wrong. We had it years earlier. But in my research, I could only find Plome, and that was only from a more recent time. So I'll say Django wins. As to what they are, the actual web application debug toolbar is a piece of code which is embedded within your web application. Once enabled, when you then access your web application from a browser, a toolbar or sidebar will be presented along with the normal page response for your web application for the URL which was visited. Through this toolbar, you can then access information about the specific web request which was just made. A range of information about the request may be presented, generally within separate panels selectable from the toolbar. This can include performance data such as how long the request took to execute, the CPU or resources consumed, or other operating system parameters. The request headers received by the web application from the browser, as well as the response headers that were returned by the web application would also generally be available. As with details of any request parameters and cookies, a range of web framework specific information could also be available. This may include session information derived from cookies, the URL route matched, and the name of the actual handler function which processed the request. Even more detailed information about work done within the context of a request for database queries and template rendering might also be available. For the case of the Django Debug toolbar, as I've been showing, it provides a set of quick setup instructions. First, you need to get the Django static files feature working. After having done that, add the Django Debug toolbar itself to the list of installed applications in the Django settings module. You'll also need to ensure that Django is running a debug mode. It's now just a simple matter of running up the Django development server and you're all ready to go, or at least you hope so. In the quick setup instructions for Django Debug toolbar, you'll find this very nice, very nice and cheery message warning you of possible impending dune. So your luck may vary, but this obviously raises the question of what is Django Debug toolbar actually doing on startup that could cause such a mess, including the possibility of circular imports and errors telling you that you've configured things incorrectly. The answer to this lies in the note in the documentation preceding this warning, and that is that Django Debug toolbar is using an evil little hack to bootstrap itself. Specifically, Django Debug toolbar wants to monkey patch additional configuration into the Django settings module and URL views. To do this, it is hijacking the validating of database models when using the development server to run code, which has nothing to do with the actual database. Now I have got windows of some changes in Django 1.7 related to application initialization, so it is possible this may no longer apply in Django 1.7. What do you do if the server does blow up on you because of this? Also, since this evil workaround is dependent on using the Django development server, how can you use the Django Debug toolbar with an alternate web server such as ModWizky? For this eventuality, Django's Debug toolbar provides you with additional explicit setup instructions as well. The first step in that is that you need to tell it in the Django settings module to not do its evil monkey patching. Now we already added the Django Debug toolbar to the list of installed applications. This is still required, otherwise Django will not know where to find the static file assets and template files that Django Debug toolbar view handlers require. Leave it out and you will immediately get a template that does not exist exception from your web application when accessed, and we don't want that. Next, we need to change the Django URL's configuration. This is to insert special Django Debug toolbar view handlers to process any request which received under the underscore debug sub URL of the Django application. Wisdom is that Django Debug toolbar should never be used on a production system, so this is also gated by checking the Django Debug flag. This is of course assuming you know why you should not be running Django in production with the debug flag, and I was a bit worried about the talks and comment on the security talk, suggesting that a lot of people do, which is a bit awkward. So after adding in the URLs for the Django Debug toolbar, we need to add in a middleware class. This is a quite important part of the Django Debug toolbar code, and its placement in the middleware classes list is also very important. It must be placed as close to the start of the middleware class list as possible, but after any middleware that encodes the response, such as gzip middleware. Finally, although you should only be running the Django Debug toolbar in the development environment, you still need to protect who can access it. The last thing you want is someone being able to access it over the conference network while you're watching my talk here. Now the documentation does say that if you don't set this, it will default to 1.2.7.0.0.1 and column, column one, which is the IPv6. But on macOS at least, I found I've also needed to add in FE80 column, column one style of address. I don't really understand the difference to the stupid IPv6 drill. I think I have to do it when I've got my VPN going or something. If someone can tell me, please do. So back to the Django development server, and we should be all good again. The evil hack of bootstrapping via Django database model verification is gone, as should those risks warned about of module import cycles and strange exceptions. What if you want to use a real production-grade whiskey server, such as modWiskey? In this example, I will make use of modWiskey express. You start off by installing modWiskey by running pip install modWiskey. If using macOS, it should all install with no issues. If you're on Linux, then you may find you first have to install the Apache development package for your specific variant of Linux, because that isn't generally installed by default. As long as you've got your package package in, that should just default, and you've got modWiskey then install. So once we install, we can now test that the package was installed correctly, and that Apache can start up properly. For this, we run modWiskey express command and give it the start server command. Normally, we would provide an additional argument with the whiskey script file, but we leave that off here because I just want to test the installation work. So in the second window, we can then open up the browser on the URL for the instance of Apache modWiskey. We've just started. What we should get is a helpful splash screen so you know it's working okay. As I already mentioned, to run it with an actual whiskey application, at this point, you can supply the path to the whiskey script file. You can also use the minus minus help option to get information on all of the command line options in case you want to change the listener port or specify a directory full of static assets. If you want some high-level information on modWiskey express, which you've never seen before, go look at PyPy, PyPI, sorry, get killed if I say PyPy, PyPI and look at modWiskey there and you'll find some information there. Now, because we're using Django though, rather than pointing modWiskey express direct at a whiskey script file, it's better to integrate modWiskey with Django itself. That way, when we run modWiskey express, it also know about the Django specific requirements for static files. What we therefore do is add modWiskey.server into the Django installed applications list of the Django settings file. Django does these days have the static files application. However, if using a whiskey server integrated with a real web server, such as the case with modWiskey, it's really much better to use that real web server. We therefore set up a location for the static files to reside and run the Django collect static management commands to collect together all the static files from the different installed applications specified in the Django settings module, including Django debug toolbar. Whatever you do though, do not remove Django contru static files from installed apps. This is because the Django debug toolbar code doesn't want to trust and I know what I'm doing and will cause everything to fail if it doesn't think that static files app will be serving up those files. So we're going to trick it a little bit, just not take it away. Now finally, here's the cool part. To actually run modWiskey with Django, and this is going to run up a full Apache, we simply run the run modWiskey management command. So this will automatically generate an Apache configuration for you on the fly dynamically for this instance. This will include mounting the Django application, setting up static files and so on. Because we're in development environment, we can even enable automated code reloading for good measure and modWiskey express will worry about all the details of getting that to work as well. So here's a bonus for this talk. You get to learn about modWiskey express, but that's not what we're here for. So the app will be running, invoking the one modWiskey command will give you the URL to then access the site as well as location of the Apache configuration which was generated in case you do want to poke around and learn from what I've done in generating it. And the Apache error log in case something does go wrong and you need to look at the application long to track down any details and exceptions that were logged in. Most important to what this talk is about, the Django debug toolbar will now also work. And that's because we followed those explicit and set up instructions. So what we learned from the explicit configuration steps requires that Django debug toolbar is dependent on a number of key things in order to work. There are a bunch of static file assets that HTML, CSS, JavaScript and images. It registers a series of view handlers which are used to produce data for each toolbar panel, which in turn potentially require page templates to render responses. Finally, it requires a Django middleware to be installed. Is this last thing, the Django debug middleware which is the key to Django debug toolbar working, is the middleware which allows it to hook into the different phases of handling a request in Django. For the Django debug toolbar, it hooks into the process view and process response phases for Django request handling. This allows it to perform actions at the start of the request, prior to the target view handler being invoked to handle a request, and finally allows it to perform actions at the end of the request as well as actually manipulate the response return from a request. Note that this is for any request, not just requests made against the Django debug toolbar's own view handlers. The part which is initially of most interest in the middleware occurs in the process response phase It is in this phase where the toolbar itself is injected into the response that comes back for a request. Before they can be done though, it is necessary to first determine whether the page is in fact HTML, and that it makes sense to insert it. There is no point trying to insert the toolbar into images or a plain text file. We're only interested in HTML. We also want to avoid compressed data or stream responses. And this is where it was important where the placement this was. If we put it after compression has been done, then this won't actually do anything. If it is okay to be modifying the response, then the rendered HTML corresponding to the toolbar itself will be injected into the original HTML response for the request. This injection of the toolbar HTML is done just prior to the existing closing body tag. The combination of that HTML as well as the style sheets and JavaScript code, which it in turn loads, when rendered in the browser results in the sidebar appearing alongside the content of the original request. As shown before, the sidebar under the time category will show how long the request took to be handled, or at least up until the time that the toolbar HTML was inserted into the response. When we click on the time category in the sidebar though, only then will the debug toolbar panel be displayed with more detailed timing information related to the request. Although the sidebar itself came back with the original request, the panel displayed as the result of a separate AJAX request made as a result of clicking in that category in the toolbar. Given it is a separate request to get the additional data, where is the data stored and how is it identified? The answer is that any JLA generator from request is saved away at the time of the request. In the toolbar HTML that comes back with the response, it has a storage ID field which identifies the item in the cache for the original request. When drilling down, a request is made with that storage ID along with the ID of the panel, corresponding to the category selected on the sidebar, which should then display the full data from the original request. Now we can't say the full details on every request ever made, so any data stored in the cache must have a finite lifetime. By default, the number of requests for which data is retained is 10. This is actually quite a big limitation as what it means is that the debug toolbar is not very useful when you're trying to use it on an application, which is any traffic beyond just that which you are manually creating with your browser. This is because any data on the request you're interested in would have been thrown away by the time you drilled down to it otherwise. As well as potentially using data, if there are other users concurrently using the same web application, the data can be purged if you are working on a page which is a heavy JavaScript component, which makes additional Ajax requests after the initial page response has been returned. These Ajax requests are a problem in themselves, and this is because Django Debug toolbar only provides you the ability to debug the primary HTML request. There is no ability to get details of any of those subsequent Ajax requests made from that page. The limitation on Ajax requests is that the toolbar is displayed in line with the current request only. To better understand what I mean by this is it actually helps to look at how the debug toolbar for the Pyramid web framework works in comparison to that for Django. In the Pyramid debug toolbar, you do not get a sidebar for the current request. Instead, you only get an icon indicating that the debug toolbar is operating. When the icon is clicked on it, it then goes off to a completely separate window. So what the Pyramid debug toolbar does is not show any details in line to the current request. Instead, you always access details of requests from a separate window, which shows a list of the last 20 requests. When clicking on the inline icon within the initial page response, it will push you across to the details of that specific request in the separate window. Important is that since you now see all recent requests, you can also see any separate Ajax requests served by the web application for other assets associated with the page. Having jumped a look at Pyramid, let's look at the history of debug toolbar for Python. As I already said, as far as I know, it started out with the Django debug toolbar. This was summarily copied in creating the Flask Debug toolbar. From the Flask Debug toolbar, then came further copies tailored for Bottle, Web.py and Pyramid. As far as I know, with the exception of Pyramid, they all simply copied the inline style of the debug toolbar as originally implemented by Django. Only Pyramid branched out and tried to address tracking of Ajax requests. So Pyramid Debug toolbar was therefore an improvement, but in my mind, they all have one big failing. That is that they are all distinct implementations. That is that the Django implementation is intrinsically wedded to the Django middleware infrastructure, with implementations of panels implemented as Django view handlers using Django templates. Flask uses its own middleware in Ginger 2, and so on for the others. We've each been ported to their specific way of doing things. The opportunity missed here as far as I'm concerned is that all these different frameworks at their lowest levels implement the whiskey specification. Yes, the debug toolbars do offer some panels which are framework specific, but the core functionality can be implemented as a whiskey middleware. There's therefore been quite a lot of effort wasting creating these separate implementations. Whereas if it was done at the whiskey level, there could be one debug toolbar to rule them all, which works everywhere. So imagine it for a moment, one debug toolbar that was best to breed when someone enhanced it due to what they found when using a specific web framework. All other web framework could benefit as well. And wouldn't that be wonderful? Unfortunately, Python web communities have a history of not cooperating, each believing their framework is the only true way. And Django unfortunately is no different. So what eventually happens therefore is that they build things which will only work with their specific framework. Someone annoying. So I think I believe we can do better. So since I think it was such a great idea, let's have a look at implementing the basic mechanisms of the debug toolbar code at the whiskey level, using a whiskey middleware. In particular, let's look at how we can manage to inject the toolbar HTML code into the response for a request. We can work on how to do that than we will on our way. This is where we can learn from the Django middleware code I showed earlier. It may not be obvious, but there are a couple of key things which are going on here. The first is that the HTML code is being inserted just prior to the closing body tag. And the second is that the content link for the response is being updated. In other words, it needs to operate on a complete response. This presents us with a bit of a problem. And that is that in whiskey applications, the response doesn't need to come back as one byte string, but as a sequence of byte strings. As far as inserting something at the end of the body, we could just look at each separate byte string until we find the closing tag for the body and then insert it. The issue is though that we also need to update the content link. And if we can't know the length of what is being inserted until it is inserted, we can't adjust the content link at the point the headers are sent and still stream the response. Now, we could buffer all the response content in the middleware and then operate on the complete response content. Then send it on after adjusting the content link. Technically though, you're not meant to do this in whiskey middleware according to the whispies specification. Plus it would cause applications which slowly stream data over time to break as nothing would be sent until all the content was generated. The actual size of the response content could also be a problem if for example one was streaming a very large file and you couldn't actually fit into memory. If having to insert at the end of the body is such a problem, why are we inserting at that point anyway? The reason is that part of what is being inserted consists of JavaScript code. And reason is that this should always be inserted at the end of body so as not to delay page loading by doing it earlier in the page. That may well have been the case once upon a time but are there now better ways of doing it which would allow us to instead insert it at the start of the response content instead. With the modern browsers we use today rather than needing to defer things to the end of the body, we can instead load the JavaScript from the head element of the page marking that it should be loaded asynchronously rather than inline to the request. By being asynchronous, it will not delay the loading of the remainder of the page. When the JavaScript is loaded, we then need to trigger an action to go back and modify the DOM for the loaded page to insert what we wanted there in the first place. We have to be careful in doing this though as we likely need to ensure that the page is actually being completely loaded. What we can do therefore is triggering the action to modify the DOM off the document being ready in the JavaScript code we are loading. How does this help though? The reason is that we no longer have to buffer up the complete response in order to be able to insert into the end of the body. At worst, we have to buffer up to the start of the body, but the amount we have to buffer up is a lot less and so doesn't present an issue with memory. It is also highly unlikely, even with the streaming response, that the head element would be sent slowly in parts. Instead, the head would be sent in one go upfront and it would only be the body which was sent out slowly. We can handle insertion but what about the UI itself? The problem with the existing toolbars is that the UI is intrinsically linked to the URL routing request handler and page template systems of those specific frameworks. My suggested solution for this is only to have a REST API served up via the Whiskey middleware for accessing any data stored about requests. Excuse me. The UI would then be a JavaScript interface rendered in the client by using a REST API in the Whiskey middleware to get access to data. We're then separated from the specifics of particular web framework's way of doing things. So if you thought I'm coming to this talk that you would learn about how debug toolbars work, hopefully you have in part but what I really want to pitch is the idea that our current debug toolbar implementations could do a big overhaul. In particular, there is a great opportunity to come up with a debug toolbar which could work with any Whiskey framework. I think we could build a better ecosystem and make our job of debugging Python web vacations a lot easier. And more importantly, diverse sets of people from different web frameworks can contribute to this so that we all get benefit from it. So one thing I personally like to do is integrate into the debug toolbar features to allow you to view additional performance monitoring data. This is where I'm coming from. So as an example, I implemented monitoring for a modWiskey but which used a New Relic platform plugin system to do visualization. It was free to use that feature in New Relic but when I announced it on the modWiskey mailing list I got zero users. Not very successful. People objected to it not being able to be self-hosted. What I don't want to have to do though is develop a separate UI for half a dozen different framework-specific debug toolbars to provide the visualization. I've also been working on a side project related to the New Relic Python agent project I work on as my day job. And New Relic is a wonderful tool for web application performance monitoring and you really all should be using it. But what I would like to do is bring some aspects of what we do with New Relic into a local developer mode integrated into a debug toolbar environment because we have a production capable way of monitoring your web application. So why not try and make use of it in a development environment as well. So in continuing this idea though there is actually in continuing this idea there's actually a bit of a twist and the problem of how to insert hasty mel into responses is something that is already solved by the New Relic Python agent across frameworks as we use that exact technique in order to implement and use a monitoring. The fault therefore is that since we do not do that already if having a local developer mode for the New Relic Python agent why not use it as the base for a debug toolbar as well. We can even have a variant of the debug toolbar you could use when using New Relic in production. So there are no other third party tools which can be brought under the same umbrella. So Simon Wilson of Eventbrite demonstrated a PyCon AU this year a debug toolbar like tool called TikiBar. This actually embodies many of the same qualities I want to achieve in pursuing this whole idea. This includes the idea I have that one could come up with a limited toolbar which works for production systems. TikiBar is though again bound specifically to Django and currently Eventbrite's own software which is exactly what I don't want to see. So I don't know yet if I'll be allowed to make it available but what I'm pursuing is a local developer mode for the New Relic Python agent that will give you a subset of its functionality for use in the development environment using its mechanism for HTML injection to implement a debug toolbar. Panels for this debug toolbar system can then provide access to the New Relic data as well as other data that debug toolbars presently provide which are framework specific. On top of that I want a mini toolbar which can deliver up safe information in the production environment. So my reason for doing all this is because I hate to see all that wasted effort be done by different developers for each different web framework or in-house system. This doesn't make a great deal of sense to me. So hopefully you're all very altruistic but I do have to admit there is a bit of a selfish aspect to this as well. And that is what I think that what New Relic does is awesome and the debug toolbar provides possibly a way for me of putting New Relic in front of you in a local environment as well so you get an idea of what we can do and how it can then be useful in your productions. Hopefully you won't concentrate too much on that stuff and you'll say, hey, this is a great idea anyway. So certainly I'd love if you go and try New Relic but if you aren't interested in that at least but you are interested in improving on the existing web application debug toolbars available then come and talk to me. Tell me what your own thoughts and ideas and features are that you want to see in this sort of thing. I'll be around for the sprints and we'll be working on this or on modWiskey. So if you come and talk to me about it then. And also finally, if you are looking for an actual paid job and you know a bit about New Relic and you're interested in that sort of stuff we are actually looking for someone who is strong in Python testing. So also come and talk to me if you're interested in that as well. The job for that would be in Portland. No remote, sorry. And that is all.