 Okay, thanks everyone. So this talk is going to go about 25 minutes. So if you have questions, save them for afterwards, I'll just be up here and we can talk afterwards. I don't think we're going to have too much time afterwards for questions. So this talk is based on a website that I've been creating for the past year and a half, just a show of hands. How many people have read fullstackpython.com before? Oh, wow. Okay. Only a few hands. Okay. That's great. Maybe you guys will be introduced to this resource, which I've spent a significant amount of time on, and it explains all the layers of the Python stack if you're deploying Python web applications. So often the reason why I created this website was because there were so many times I would talk to new developers, even if they had worked in a different language, Ruby or Java, and they were wondering how do I create a Python website, a web application, and then deploy it, put it out on the Internet. So they were learning the Python syntax. They were starting to understand, okay, I need to pick a framework. If I'm not going to write a framework myself and I want to create a web application, I'll pick something like Django or Flask. I'll work with Python, learn the syntax, and iterate on my concept. And so many people have ideas for what they want to create for their web application. And they create that, the initial concept, and they say, now I want to put this in the hands of other people. How do I go about doing that? And you start explaining things like, well, you need to set up a Whiskey server, and then you need to have a virtual private server, and you're working with them, and you're drawing a diagram, and all of a sudden it ends up looking something like this, which to someone who is new to web development can be very intimidating. And so this diagram here, if it's difficult to see, you can actually just go ahead and download that. That's on the front page of fullstackpython.com right now, and there's a PDF of it. So if you want to zoom in and click on the links to different sections, certainly go ahead and do that. We'll go back to this page for a moment here. It's very intimidating for someone who is new or coming from a different language. I moved over to Python several years ago from the Java world, and there were all these new terms that, even though I was an experienced programmer, I was still questioning, what do these things mean? And so when I'm working with new developers and explaining to them these concepts, what I say is, OK, this is sort of the end state. Let's wipe the slate clean for now, and let's start with what you've been working with, what have you been working on. So if we take away all that stuff for the moment, and we'll explain it piece by piece how you deploy a Python web application, you start with this Whiskey framework, or most just call them web frameworks. And this is what it looks like. So if we take a look from your perspective, if we're looking at this, the clients are on the left side. You may have a mobile application that's accessing your application programming interfaces, APIs, other applications that are doing machine to machine communication with JSON or XML, and then you have the traditional web applications, people accessing your web application on the desktop or the browser or just through the mobile web. I've labeled the middle box internet. It's a little bit deceptive here because while certainly a lot of communication is going on on the internet, this may also represent things you're hosting yourself. So if you're hosting one of your own services, it's also technically on the internet, on a different server, you're communicating to it through the internet, but it's all lumped together in that center column. And then the one on the right is your web application deployment. What components do you need in order to get your site live and running and understand how people are using it? So let's start with the web framework. The thing that most people start with is Django because it's just recommended, hey, Django's got great documentation. It'll walk you through the initial polls application. You can build a blog with it. There's also been some popularity with Flask now. And so these are probably the two big frameworks as far as what people are recommending to new developers in Python. There are certainly other great frameworks out there, and some people start with them, Bottle, Pyramid, all those. But for right now, these are the ones that seems like it comes up over and over again. Start with Django or start with Flask. You can move to something else later. These two frameworks represent opposite ends of the spectrum. On one side, you have Django, which says, batteries included. Everything you want to do when you're creating a web application, you should be able to do with the framework, the core framework itself. On the other end, the other end of the spectrum is Flask, which says, let's have a small core and a lot of extensibility, make it really easy to add extensions to the framework itself. You see this in, for one example, user authentication. Not every web application needs user authentication. So Flask does not have an official user authentication module built into core. There are many extensions that will help you do that, like Flask login, but that's not built into the core itself. It's a separate piece that you would install as a separate dependency, whereas Django has an official way to do user authentication. So these represent these opposite ends of the spectrum. Now, the great part is we don't really have to choose which framework we want to run with which web server, or I'm sorry, Whiskey server. And the reason for that is because of Whiskey, the way that this was outlined. Whiskey stands for web server gateway interface. And it is a PEP standard, a Python enhanced proposal standard, which we'll talk about in a second. The way that we would look at this from our deployment diagram is the Whiskey server is what is going to be executing our Python code, whether that is the code that we've written, or the framework or other dependencies that we have. Again, this is a PEP standard, 333, and then 3333. The second one was added for Python 3 support, mostly just deals with some Unicode, but it basically says the 3333, if you're doing that already, you're still using Python 2, you're good, you're fine. The second one was just added because there's some changes with Unicode and Python 3 you need to adhere to in order for you to be adhering to the Whiskey PEP standards. Here's how this works, how Whiskey works. And it is actually a very simple, I recommend if you are doing Python web development to read these standards. They're actually very readable, especially this one in particular. And what it's saying in 3333 and the second one is you have a web server, or also called a gateway, sometimes also called a Whiskey container, and it is invoking a callable object on the Whiskey application, also sometimes called the framework. Those terms are used interchangeably within those PEP standards. And the reason for this is because before we had these standards, before we had Whiskey, we did not have a uniform way to execute Python code on the server. And so you could not get one Python container to work with another framework interchangeably. Now we can because of this standard. This is an interface standard, which the servers and the frameworks implement. You may have heard some of these Whiskey servers, rather. Green Unicorn is one of them. ModWiskey, if you're working with Apache. And Mewiskey, MicroWiskey, UWiskey, someone needs to clarify exactly how you pronounce this one. But this is one of the original ones that was created for executing Python code. Now we need a place to deploy our application. We understand what a Whiskey container is. It executes our code. We understand that we're creating a web application, a Python web application, in a Whiskey framework. And we need a place to put it. And that's where servers and an operating system come in. So this large rectangle we have here on the right side is our server. Or you may have heard Platform as a Service. We can use those somewhat interchangeably. There's a few different variations of this. One is just a bare metal server. If we take this laptop on the front row here and we hook it up to the internet and we have a static IP address, which doesn't change over time, and we point our domain name system to that laptop, we can run our web application off of that. That's all we need. That's often considered a bare metal server, or just a bunch of servers that we have in a closet somewhere. There's also virtualized servers, which means we have an actual hardware server, and then we're virtualizing many operating systems on top of that larger server. There's infrastructures as a service. That's Amazon Web Services, Rackspace, Azure, IBM Bluemix, and Platform as a Service, Heroku, Python, anywhere. For the purposes of this talk, we're just going to be talking about, just think about, if you have a bare metal server in your closet somewhere or a virtualized server that you've rented from Linode or DigitalOcean, one of those services. We're going to be running Linux on top of it, some sort of variation of Linux. I personally use Ubuntu all the time. CentOS is great. Red Hat, there's all sorts of different distributions. It matters. What matters is, does it work for you? But ultimately, we're going to be using Linux for our deployment. So that's another box that we would look at as being running inside of our server. Now, the one thing about this diagram here is that there are all these clients, and they're not yet talking to our web application. They need to talk to our web application through a web server. And the two web servers that are out there are Apache and Nginx. Certainly, there are others, but these are the top two ones right now out on the internet that we are most likely going to use with our Python deployment. And we would configure these so that they would pass through certain requests to the WISGI server. Let's say, for example, we write a blog application, and we decide that our domain name, whatever, example.com, forward slash blog, is going to be our Python application. We would configure the web server so that it passes all forward slash blog requests onto our WISGI server that will execute the code along with our blog. And then it will pass back the results. If this is just a web page that's creating, it's just going to pass back HTML from the WISGI server to the web server, and then that will be passed back to the client. For just generating JSON, for example, if we just have a machine-readable version of our blog, it will pass back just the JSON from the WISGI server to the web server back to the calling invoking client. Now, the thing here is, what about our code, right? So we need to get our code onto that server somehow. And that's where source control comes in. Certainly, we use source control when we're just developing our application and checking files in, but we also use it as a part of our deployment process so we can get either the latest version of our code or a specific version, a specific tag of our code onto our production or test servers. A couple of them that are mostly used now would be Git and Mercurial. They're just distributed version control systems, and that's the ones that most Python developers are using today if they're not being forced to use something that is from 20 years ago or something like that. OK, the way that this looks is, we may host our source control ourselves. We may have a Git server set up on a different repository, and we're going to pull our source code down onto our server from that source control repository. We may also use something like GitHub or Bitbucket that is just going to host it for us, and we'll pull it down from those servers instead of hosting it ourselves. And maybe we pay them money so that they can host it as opposed to us having to rent another server or build another server ourselves. When we pull down the code, we have application dependencies, which are stored in a file, most of the de facto file requirements.txt file, which will look something like this. We're using Django 1.5.4, we're using the Twilio helper library, we're using requests and Stripe, all these different libraries, we've pegged specific version numbers that we're using. Now we need to get these dependencies onto our server as well. There's two concepts here. One is virtualM, and virtualM is a way of creating an isolated Python execution environment. So it's basically the way that I like to think about it is it's just if you have a system copy of Python installed that's not going to be clean, it's your only version of Python. VirtualM gives you unlimited number of copies of your Python interpreter with their own set of dependencies. So if we're in the consulting environment, we've got three clients, one of them is using legacy Django 1.2, and then another one's using 1.4, and another one's upgrading to 1.7. We have three virtualMs, one for each of those clients, and they have all the separate dependencies, not only Django, but whatever version numbers of other libraries that they're using in each corresponding virtualM. Now to get files into the virtualM, we could technically host our own server with all the distributables for those versions, but most likely we're going to use PyPI. It's not to be confused with PyPI, PyPI is a different interpreter, which is a just-in-time interpreter. This is PyPI, which is a centralized Python hosting repository, which is at python.org. Here's how this looks. I've also added just a little sticky note when I add something new to this diagram, because the diagram does tend to get busy over time. I've just added a little sticky note to show what we're changing every time that I show this diagram as we're building it up. So in this case, we're pulling down the source code from source control, we're putting it in our virtualM, or we're putting it on our file system, it's being executed in the virtual environment, and we're pulling down our dependencies into the virtualM from PyPI, usually through something like PIP or Easy Install. We were just pulling those version dependencies down through from PyPI and putting them on our server. So now we've got this whole execution environment and the web server is handling traffic. There's like a really big missing component here, and that is the data. So we need a database, something or other. So there's two forms usually that we're storing data in. Most of the time we're storing just traditional data in a relational database, so that's the one on top. And then there's also all sorts of NoSQL databases or data stores that sometimes people don't even call them NoSQL data stores just because they're so ubiquitous now. Relational databases, most likely we're gonna be using MySQL or Postgres. I tend to use Postgres most of the time. It seems like what a lot of the Python developers are using nowadays, certainly MySQL is a valid option as well. You kind of have to be able to flip back and forth particularly for in a Python consulting environment, but if I start a new project and it's my choice, I tend to use Postgres. Now from a NoSQL perspective, most people are thinking like, okay, document-oriented store like MongoDB where I don't have the same constraints that I would have in a relational database. But this also includes things like Redis and Redis is a key value pair store that is just in memory. And that's often used for just if you have a long running database transaction and you just wanna store the aggregated results, you just put it in a key value pair store like Redis and then it'll just pull it out very quickly from memory because relational databases are going to disk to actually retrieve the data that you want. Now with something like MongoDB, there's been a lot of talk over the past few years about what NoSQL data stores can do for you. The way that I like to think about it is you're changing the where your constraints are. So instead of having a schema in a relational database which is enforcing foreign keys and you have to have just a structure to your database and you have to run database migrations, most people say, I don't wanna deal with all that stuff, I'm gonna use a document-oriented data store like MongoDB or React and I'm just gonna use that and that will free me of those constraints. The problem is that that doesn't actually remove the constraints altogether. It moves the constraints up to the application layer. So when you change your schema, sure, you're not running a migration but you have to account for that in your code and so the complexity shifts from being at the database level to the application level. So that's just something to be aware of. It doesn't mean that it's not a valid option for your use case but just think through that clearly. All right, there's a couple of technologies that are browser-based. Unfortunately for lack of time we will sort of just glaze over these JavaScript and CSS if you're building any sort of web application. This is what is going to be rendered, help your web application be rendered in the browser. CSS cascading style sheets from just a layout perspective and then JavaScript for dynamic interaction and also things like Backbone.js or Angular for these more dynamic one-page applications which seem to be a little bit more popular in certain industries now. If you are just creating an API, you really don't even need to worry about that. You're just sending machine readable JSON or XML or YAML. So you're not even going to be worrying about those front-end technologies. There's one other thing about this. It's a content delivery network. So in one setup, you could have your web server serving up these files but in another setup, you could have a content delivery network like Amazon CloudFront or Akamai to serve up these static files for you. That takes the load off of your server. So that's one option as you're trying to scale up. You can say, I don't want my Nginx or Apache server to be serving up JavaScript and CSS. I'm going to offload that to a content delivery network. One more thing to complete this is task queues. And this is often sort of forgotten. It's like something that happens like after the factory where you're like, gosh, my HTTP request response cycle is taking way too long. It's hanging on this API call. Well, you're probably, if you're doing something that takes a long time, then you probably should be doing that outside of the HTTP request response cycle. So you'd use a task queue, something like, you may have heard, salary tends to be the most common task queue. There's also Taskmaster, which is a bit lighter weight, and RQ, which is originally intended just for Redis. Okay, so what that does, that completes the right-hand side of this diagram. The task queue allows you to execute Python code outside of anyone sending an HTTP request or you responding to that HTTP with an HTTP response. So if you have a long-running database query, maybe you have to calculate all sorts of stuff across millions of records. You're gonna run that in a regularly scheduled task queue job. You're not gonna run that when someone says, give me the results of this query. You're just gonna run it say every 15 minutes or every 30 minutes. You're just gonna do it on an ad hoc basis rather than during a request response cycle. Okay, there's a few other pieces that we need to talk about here. So we've got our application up and running. Like this is a pretty standard setup for most Python web applications. Now, one thing to note is that this is very abstracted from a server perspective. Most of the time you're gonna have your database on a separate server than your web server. You may be scaling out. You may have five web servers with the whiskey servers running on top of them and they're all talking to the database. So you could take a look at this right-hand side and there's multiple ways to scale that out. We're not gonna concern ourselves with that at this point but there's a lot of great information on how you scale out these architectures. Just think of this as these are all the pieces that you need and eventually you can break them apart so that they scale better. So we've got people coming to our web application and now we wanna know what are they doing, right? So this is where web analytics comes in and it answers that all sort of encompassing philosophical question of is there anybody out there? And they give us pretty graphs and charts and numbers and stuff and we can see, oh my gosh, people are actually coming to my web application and using it. This is amazing. So we can see what are the traffic trends and if we put out a new version, are people using it more or less or how are they interacting with this? So the way that this fits in is that we would have web analytics usually hosted on another server. So Google Analytics is a third party hosting and they would take data from the browser and they would just aggregate all those metrics and create all the pretty charts for you. There's also something like Mixpanel or Kissmetrics and you can also do this now with Google Analytics as well where you can send data from your servers as well but in the traditional sense of, you basically just slap a little JavaScript line on your HTML page that's rendered and then that is going to send through JavaScript from the client side analytics and data to Google Analytics servers. So that's basically the setup that you'd use here. Okay, so now, cool, we've got like web analytics and people are using this but now we run into a problem. Like there's an error and what do we do now? So that's where logging and monitoring come in and these topics overlap to some extent the way that I like to think about it but I don't know if there is a hard definition for this. The logging piece of it is generally by events. So I want to log every time I have my 100th visitor or I wanna log when I get a certain error or I wanna log when a database query takes longer than a second. There's specific events that we're looking out for whereas on the monitoring side it's basically just monitoring passively in the background so that later on if there's a problem and you wanna see what's happening you can just look at all the data that's been passively collected over time by an agent that's running on your server. So the way that these generally fit in you can certainly host monitoring and logging solutions yourself but there's also third party ones like New Relic and Copper Egg from a monitoring standpoint and then from a logging standpoint there's Logly and Splunk and many other services like that. So you can host them yourself or you can use a third party service and this is essentially the setup that we would use. Your server is going to be sending data and then also too the browser tends to send data. You'll often have a little agent that runs in the browser that is sending data to a monitoring service just passively in the background. There's some other topics here that are important as well. We unfortunately won't really have time to dive into them but just in general we need to be thinking about what is the deployment lifecycle look like. If you pulled this up unfortunately cut off the bottom with other topics but I'll just tell you real quick what those are. Just thinking about the deployment lifecycle when we check code in should be picked up by a continuous integration server it should be text tested and then there should be some automated deployment process so we don't have to manually hand do the deployment and going along with that is configuration management tools you may have heard Puppet, Chef, Saltstack, Ansible. What that is going to allow us to do is set up the servers, the server or servers in a certain state and then automatically either revert to a certain state or evolve our state through scripts rather than just manually hand tuning things like changing Apache settings by SSHing into the server. Other things, caching is a way for improving the performance of your web application and then there's just other topics that are out there that are just certain things like well what other resources do I need in order to learn about certain frameworks besides just Python and Django. Now if you take a look at this PDF on the website each one of these is actually a link so you can go into each topic and see what the best resources and a plain text explanation for what each topic is. So going back just to the beginning when I've been working with all of these junior developers who are like I'm just getting started and with Python I'm building my application I've got this great idea and now I want to put it on the internet. A lot of times what they're looking for is just like a plain language explanation of what these topics are and I would encourage people to when you're writing open source software and you're working with these things a lot of times beginners are looking for just a plain language explanation of what something is, analogies that allow them to better understand it. So often and I get emails about this so often in the Python but also in other ecosystems in the programming world we try to make things sound really complicated when in fact if we just use some plain language it will help people much better to understand those topics. So that's what I'm trying to do with fullstackpython.com I've had about 76 or so thousand people visit so far this year which has been awesome. So thank you guys and the audience that have read that and I encourage the rest of you just if you're wondering about these topics I link to all sorts of other topics that are on there. I would love feedback on the website and also on this talk as well. My name is Matt Mackay, I'm a developer evangelist for Twilio. Here's my contact information. I have tweeted out this presentation right before I came up. Feel free to follow me on GitHub or Twitter and here is the rest of my contact information. Thank you very much. Do you want to take a couple of questions? Oh, if we have time for questions. Got any questions? No questions? Okay, thanks very much Matt. Thank you.