 Hit me OK. Lots of echo. OK. Hi, everybody. Julian, that's me on the internet. You can find me in various places. I'll try and share the slides afterwards. And I hope to leave plenty of time for questions in the end, although feel free to interrupt me as we go. So I work for a company called Magnetic. Just a 30-second background. We're in marketing tech. We use PyPy. That's why I'm here. Usually they send me to fun international places. It's actually the first conference I've ever spoken at in the US, I'm pretty sure. But it's wonderful. And like everyone else, we're hiring. So if you want to talk to me about work after this, feel free. There's a bunch of my coworkers here, too. Whoops. So as I talk about PyPy, but just for some context, we'll kind of start at the beginning. Maybe it would be worthwhile just to ask, how many people, if I say, can you tell me what the definition of a compiler and an interpreter is? How many people would be able to answer? OK. Small number, maybe a little less than half. So for those of you, this might be a little bit slow, but we'll start from scratch. So yeah, just for some background. So programs, right? So what's a program? At kind of the simplest sense, a program is basically something written in some language that takes a bunch of inputs and produces a bunch of outputs. So whatever it is, roughly, it's that. So yeah, like Google Chrome, I mean, maybe your input is like you typing stuff into the address bar and your output is like the rendering of Python.org. I apologize, that's quite small, but hopefully nothing else will end up being that small. So it's a program, I think in the case of Chrome, it's like a bunch of C and C++ stuff tied together with a whole bunch of other stuff. It's a thing written in some language. You give it some input, it produces some output. Getting kind of down to as far as we can go, like CPUs execute machine code, which is basically a bunch of binary. You pass through machine code through the CPU and it does what you tell it to. Side note, just for fun, potentially the CPU is taking that machine code and turning it into a bunch of other stuff and not actually executing the exact x86 assembly, let's say, that you're giving it. So at the end of the day, whatever your program is written in, it has to get down to whatever instruction set your CPU is going to execute. And so for our story before, where we have a program, at some point, whatever language your program is written in, you have to turn it into something that is just exactly that, like a bunch of machine code that your CPU can execute. So you still have your input, you still have your output, but you have to have some sort of machine code that you're going to run, no matter what you started with, Python or whatever other language. You have to ultimately execute what your CPU knows how to execute, which is, for most of us, like x86 assembly or whatever. So just a side note, because we got here, so interpreters and compilers, a nice sort of blurry world between the two of them. But it's relevant for Python and it's relevant for us. So Python's an interpreter. We'll talk a little bit about how it works, both in the case of Cpython and PyPy. But there's kind of a nuanced difference here between an interpreter and a compiler. When we say compiler, we'll talk about a thing that takes a piece of source code and turns it into machine code. Compilers that compile to other targets, but that's what I'll mean. And by an interpreter, we'll mean a piece of software, a program that takes some source code and turns it into something intermediate that then executes on some other higher level machine. We'll talk about some more in a minute. But Python is, in that sense, an interpreter. It's a piece of software that takes Python source code and executes it without first turning it into machine code. So you don't take Python and turn it into single binary. It's not Rust or C or any of those languages. So specifically, I'll start talking a little bit about Cpython. So Cpython is, for those of you who don't know, Cpython is what you're going to get if you just type Python and don't otherwise know you have something else. So if you downloaded something from the internet or from your package manager or from wherever else, you're getting your software. Cpython is what you got. And the reason why it's called Cpython we'll talk about in a minute, but it's basically written in C. Here's your answer. And it's a program. So back to our original definition of what a program is, it takes some input and it produces some output. In the case of Cpython and the case of interpreters in general, the input that it's taking is the source code that you wrote. So you type some Python into a file or someone gave you some. That's your input to Cpython. And the actual process that Cpython goes through to actually execute it, there is a step in the middle there, which again comes back to our same conversation from before. So Cpython takes the source code that you gave it, turns it into some byte code, which we'll see in a minute. And then it takes that byte code and executes it on a VM. And a VM is basically, you can think of it as a virtual CPU. So our actual CPU, we make believe, let's say, it takes x86 assembly or whatever assembly language your CPU takes. A VM is a fake CPU implemented in software. So it's kind of a giant loop. We'll talk about it in a minute. And that giant loop takes an instruction set. And that instruction set is basically going to be used to execute our Python code. And if you look at Cpython, if you've ever pulled apart the Cpython source, there's a file in the Cpython source tree called like C of L dot C. And in there is basically a giant, I think in Python 3, it changed a little bit. But in Py2, it was basically like a giant case statement. And each of the cases in the case statement is like one of the byte codes that the VM accepts. And so for every, Gabe, you have a password? We're about to find out. I'm glad you do. If you didn't have a password, I would yell at you. There you go. So in Cpython, the byte code instructions that we're about to see get passed through this loop, which basically just switch cases over each of the byte codes that flows through the interpreter and executes them one by one. And if you look in C of L dot C, you'll see implementations for each of the byte code instructions that we'll see. So I'll show you some examples in a minute. As soon as we get back plugged in here. Any questions so far? Good. So here's this kind of dumb little function. So I would assume a lot of you have probably seen both halves of this. But again, just for background, we'll go through both. So on the left here is some Python source code. Kind of a strange function, but just for a bit of demonstration. So it takes two numbers. If x is bigger than y, then it adds them. Otherwise, it subtracts them. Why? I don't know. So there's this thing called the disk module, which you might have played with before. Basically, it's a module that can take a bunch of different kinds of objects and show you human readable and not so human readable lists of the byte code that that code object will actually execute. In this case, the functions code object. So if we pass in the function, we can kind of read this from the top. Can I ask you to put this on your desk? OK. So if we just read this from the top, it's actually not that hard to read. It's somewhat typical of kind of like high level VMs. So load fast is basically a byte code instruction that loads a variable. So we're going to kind of compile this in our head and see how that compares to the byte code on the right. So if you think about how this function is going to execute, the first thing that it does really is it needs to know what's the value of x and what's the value of y. And those are variables that were kind of like passed into this function, so it needs to basically look them up. Compare op is basically an instruction that means take the two things that are on the stack, which I haven't defined. But basically, you can think of a stack if you don't know what it is already as basically being the place where we're storing the stuff that we're processing. So compare op means take the stuff that's on the stack, compare the two of them. This one takes an argument. So compare op 0 means do less than, which is what this code is trying to do in reverse, actually. And then pop jump if false means this byte code told us whether x and y, whether x was less than y. And if that was false, then we're going to jump to instruction 20, which if you look at the line numbers down here, 20 is going to be the bottom of this loop, which corresponds to there. And so I can kind of read through the rest of these. But I hope it's somewhat clear so far that what's happening here is there's basically some Python source code. It's been turned into something somewhat resembling assembly, if you've seen it before. And each of these byte code instructions kind of like is instructing this VM in C Python to execute a bunch of slightly lower level, but still fairly high level instructions. You're going to run the slides, good. So C Python, again, it's written in C. It's a byte code compiler and a stack-based VM. That's a bunch of words that mean it produces the byte code on the right and then executes it on top of a stack machine. And then I'm being slightly not fair, but roughly C Python's design sort of prioritizes simplicity of implementation. So the C Python implementation is basically trying to be fairly simple and fairly accessible. Actually, from what I hear, I'm not a C programmer. From what I hear, if you read it, it's half decent C code, fairly well commented, and et cetera, et cetera. But those byte code instructions that we had on there, they correspond to many potential actual machine code instructions. So we had that load fast there, which is like, hey, give me the value of some variable. And in my imaginary quite fantastic world where CPUs take like six-bit instructions for some reason, that corresponds to potentially like 10 instructions, which if you can think in your head, possibly what those are doing is like, load fast means get me the value of some variable. Maybe I need to go look that up in some array of where I'm keeping all my values of variables. So you need to implement that using some lower level language, which we said in the case of C Python is C. And in order to actually implement that byte code instruction, which is tell me the value of this variable, there might be potentially more than one assembly instruction or machine code instruction that actually needs to execute in order to perform that operation. And so in the general case, you have on the left all of the byte code that your C Python VM needs to execute. And on the right, for every one byte code, there's potentially n number of assembly instructions that might actually need to execute in order to run that one piece of byte code. Now I'm being slightly disingenuous when I hit the next slide and I say, OK. So if I done slow, right? Because for every byte code instruction that you want to execute, you have to execute, let's say, 30 machine code instructions just to execute one piece of byte code. Whereas if you're using some sort of compiled language, you would sort of get the minimal set of assembly that you needed to execute because there wouldn't be this middle layer. And that's kind of the punch line in sort of this background information is like ultimately when you have a VM, there is a middle layer in there. And the middle layer is the thing that's executing your byte code. And so if you get rid of that, you're faster, right? So that's a bit of a lie to begin with. And I don't actually mean to tell it in this case. So because of all sorts of fun complications and even some non-complications, sometimes actually a VM tends to be fast because of branch predictions and cache missing and all sorts of other fun low-level things. So I don't know about those things. You can probably talk to people who are more low-level programmers than I am. And they'll tell you about them. But don't assume immediately that just because you have a middle layer that it's going to be way slower anyways. Sometimes regularity is good for your computer. But even besides all that. So no, we're not saying Python is slow. First of all, in the pedantic sense, we've just been describing Cpython so far and we're about to talk about PyPy. But second of all, if you think about what we've talked about so far, all of that applies when you're actually executing computation. So byte codes are like, I need to compute something. If you're not doing computation, you're fine anyways. If you need to wait on the network, you need to do something else, you're going to be fine. The kind of basic point that we were making where every piece of byte code might correspond to more than one potential machine code instruction and apply if you're not actually executing byte code. So certainly don't draw the conclusion so far that Python is slow, but we can do a little bit better in terms of just raw CPU utilization speed. So why can't we do better by just doing the normal thing? The normal thing being, let's say, you know your C. How does a C compiler work? Again, it takes the source code and produces some machine code. So why can't we just do that for Python? Why can't we just take the Python source code and produce some machine code in the same way as other compiled languages might do? Why do we need any sort of novel solution? Let's just do the same thing that everyone else is doing. So it turns out that that's somewhere between very hard and nonsensical. So take a function that looks like this, for example. What assembly do you want to compile this particular function to? So if you hit Next. So there isn't assembly instruction that does add a bunch of registers. So this assembly instruction machine code will basically take a couple of machine ints, I think, into registers and add them together, put them somewhere on some other register, I assume. So there is an assembly instruction for adding two integers. But that Python code doesn't add integers. It adds whatever you gave it. So x and y in that function, you pass in two integers. It's going to be integer addition. You pass in two strings. It's going to be string addition. You pass in two random custom user-made objects. And it's going to be whatever those objects say addition means for them. So that's just a function on its own. If you're a compiler and you see that function, how do you know literally what do you generate for what you're seeing in front of you? We'll talk a little bit later about some existing specialized compilers for Python that actually solve that problem in somewhat obvious ways. If in maybe some specific case, you say, OK, but I'm using this function for integers, and then you know what to generate. But in a general case, this assembly code and the Python code on the last page are just nowhere near each other. The Python code is like I add two objects, and they tell me how they want to be added. And this assembly code is like I have two machine words in two registers and add the two of those. There's just no way to reconcile the two things. And it gets even worse if you have something like this. This function doesn't even... The types of this object are on some server somewhere. Whatever... If that JSON object has two ints in it, then this is integer addition. If that JSON object has two strings in it, it's a string addition. You can't hope for a compiler to be able to tell what to do with the types of those two things. So we're in a bit of trouble here. And I don't mean to kind of set up things as like types are the only thing that we're gonna run into trouble with, but certainly it's a big hurdle to get across. It's like this function, I don't know how to type the objects that we're gonna be working with, let alone know how to convert that into some sort of machine code that I'm gonna be able to compile into. So if we get back to sort of like the crux of the problem here or what might be the problem, my program's too slow, and you're telling me we can't write a compiler, so what's next? So the punchline is PyPy, and we'll talk about that in a minute, but certainly before PyPy exists, and even to some extent in some circles now, the old answer was like, okay, so you need to change it in some way. So your program's slow, you wrote it in Python, change it somehow. And there were sort of like three, at least three paths, these are kind of the most obvious ones. One of them was like take a piece of it, rewrite it in some other language like C usually, and then use let's say the C Python C API, which is a way that you can take a bunch of random C code and kind of like use it from within Python. So go take your Python code that was slow before, write it in C, then of course you are kind of like statically typed, so you do need to say what all the types of your stuff is. And then you run it through your normal C compiler, out pops a C extension module, you load that from Python, and basically you just rewrote a section of your code in Python that was in Python into C. So your code still behaves the same, but you changed it to basically be written in some other language and then plug that piece back in. And there was an okay solution. C Python has API has a lot of problems, and again from people smarter than me, it has a lot of problems that are quite hard or impossible to fix in terms of like a compatibility standpoint, like the C Python API exists and people use it, and a lot of things in there are kind of questionable, I think, but it certainly an option, it certainly works. Cyton was another option, Cyton for those of you who haven't used it, it's basically if C and Python had a kind of bastard child that would probably end up looking like Cyton, again in that kind of like, Cyton's a good thing, at least in the general sense, like, but that really is what it is. It's basically like it looks like Python with a bunch of C sprinkled through, and there are some additional mechanics that like give you tighter interrupt, but basically it's a thing that kind of looks like Python with a bunch of like types mixed in and some other functionality to like import stuff from C if you actually feel like doing that and a couple other things. And then that thing ends up getting compiled into, again, an extension module that you can load. So this is sort of like using the C Python API as a bit more portable in some sense, and it looks a lot more like the Python that you started with, so in some ways this is changing your code less, and certainly a lot of people do still use Cyton, and from what I hear, some people actually tell me that they have quite a lot of reason to keep using Cyton. So that was certainly another reasonable avenue to pursue. If your program's too slow, you take a chunk of it and again you change it, you write it in Cyton, and you plug it back in again. And then of course kind of like the most drastic or easy solution depending on your perspective is like just wholesale rip out a portion of your program, rewrite it in some other language, and communicate back and forth with some like inter-process communication. So like your program's too slow, great, like take this section of it that's doing computationally intensive stuff, move it into some language that's more suitable for computationally intensive stuff, and then communicate back and forth over whatever method you'd like. So these were the three historical choices. Again, the punchline here is PyPy, but kind of the thing that's getting us to the punchline is a JIT. So a JIT, Justin Time Compiler, is gonna be our way, PyPy's way of basically fixing some of the kind of troublesome problems that we were running into before about like how do I compile programs that I don't know enough about. Justin Time Compiler's the Justin Time being basically at runtime. So while your program is running, it's also potentially doing compilation. That's kind of like the piece of the puzzle that's like okay, you don't have enough information ahead of time to know what you might need to generate, but maybe at runtime it'll be better. Maybe while your program's executing, it'll be easier to figure out what I need to know, and maybe even at runtime when I have more information, I can do even more intelligent things that I could have ahead of time. So we're kind of partitioning the world of compilers now so that we had our languages that we were discussing before, which are kind of like you take a compiler, you plug in the source code, out pops machine code, because that's what you need to execute. Into this world now we're like, okay, we can't do that ahead of time. We're gonna do that at runtime. We still wanna end up with some machine code because that's what's gonna end up being fast. We're gonna get rid of that middle layer, but we're gonna do it at runtime when we have enough information. And so that's what PyPy is basically. It's many things, sort of the most pertinent one for this discussion is that. So in kind of like correspondence with what we were talking about before, it's written in a language called Rpython. I'll mention a little bit more about that in a minute. Again, it's a program, so it has to be written in something, right? So it's a program that takes in a bunch of input, again being Python source code and out pops, whatever output your source code actually performs. So it's written in Rpython, very similar implementation to Cpython at kind of the high level. It again has a bytecode compiler, which means it takes your source code, turns it into some bytecode and executes that bytecode on a VM. The difference is gonna be that every once in a while it decides not to do that and to do something entirely different instead, which will be like produce some machine code and execute that instead of executing whatever bytecode corresponds to your source code. And most importantly, it interprets the same exact language. So we're gonna kind of get exactly where we wanna go here. Before I kind of specifically say, so Rpython, just another word on what Rpython is, the R stands for restricted. It's a language. It's statically typed and it's amenable to being like annotated with types as part of like the translation process. And it's also part of this like larger tool chain that can be used to write other interpreters other than PyPy. But it's basically Python. It's Python with static types or slightly more kiddingly what the PyPy guys basically say is like, it's this thing that accepts whatever subset of Python the translation tool chain doesn't error out on. So it's roughly Python with enough information to be able to statically or to be able to pre-compile just the interpreter part. So, yes. So the last sentence was Rpython that PyPy is written in is basically a subset small enough of Python that it can be translated into machine code. So it has none of the problems that we mentioned before like you know or it, the tool chain knows enough about what the types of everything is to produce machine code ahead of time. And roughly that's the important part. So, you run your Python code with Cpython, assuming that's Cpython, you run the same code with PyPy and sort of the most important part on this avenue is like, okay, we finally, we didn't have to actually change anything in my code. Like I wanted stuff to be fast but I didn't want to learn some other language and PyPy is kind of promising to be exactly that. Like you take some code that you had before you run it through PyPy, it's faster. You didn't need to do any extra work. So, Jits were kind of like the vehicle for how we got enough information. The philosophy is something like this. So a lot of the trouble that you run into in Python boils down to stuff like this where you can do a lot of stuff but some of the stuff that you can do is kind of unlikely in like a probabilistic sense or maybe not quite that scientific but like in Python you can do a lot of crazy dynamic stuff but you're very unlikely to actually be doing that crazy dynamic stuff and so if you can kind of like make some assumptions that nothing crazy is going on you can kind of like get rid of a lot of the overhead that you would have otherwise had to have dealt with from just kind of like what the language allows you to do perspective. So we spoke about one example before about like just passing in whatever objects you feel like. If you had a program where kind of like imagine as like an external observer which is really what the jit is you're kind of watching this function being called and you see it like 20,000 times being called with two integers. Now it's very possible for your program all of a sudden to start calling it with a bunch of strings instead but from a probabilistic point of view if you see it like 20,000 times you're gonna say you know what probably this is going to be called with integers if it turns out something else happens you'll need to deal with that situation because of course you can't sacrifice the program doing the right thing to make it fast but at least from kind of like a optimization standpoint or like a what can I assume about this program it's probably reasonable to assume that what you're seeing so far is what you're gonna continue to see until you see otherwise. Same thing here for just like literally attribute access. So attribute access is very expensive in Python. See Python. Why is that because like if you want to attribute if you want to access some attribute off of an object you've probably seen like most objects have dictionaries that actually map their attribute names to the values and you can't get rid of that dictionary really besides slots. You can't really get rid of that dictionary because someone can come along to your object and just add random attributes to it. Like your object didn't have an attribute called like blah or whatever. Someone can come along at some point in the program and just plant another attribute on your object and the object needs to keep track of that additional attribute so that like everything needs to know like where's that attribute living in memory. You can't just kind of like allocate a bunch of memory and say like here this is all I need for this object and I can just kind of look up attributes by just indexing into that memory. You need to actually maintain a whole bunch of other information because you know someone comes along and adds a bunch of new attributes you're gonna need to be able to put them somewhere. So even something as like completely simple as accessing an attribute in CPython like other than slots you need to actually do a hash lookup to actually figure out you know what the value of that attribute is and that's expensive. Whereas if we can make the assumption that we made before and we say like you know what some things actually the language allows you to do but they're not likely to happen so an object like changing types. Like there was a function example. There was a function that every time we see it it has the same types going through it. Yeah I mean it's possible for all of a sudden new types to start coming through but it's let's say unlikely or changing attributes. So classes have some or instances have some number of attributes. It's possible for all of a sudden that instance to have like a whole bunch of different attributes than the ones that it had before but it's not very likely generally speaking objects keep their attributes over their lifetime not necessarily attribute values but at least like the set of attribute names. At least that's constant enough over the lifetime that maybe we can make an assumption about that. And if it's not true there'll be mechanisms in PyPy which I won't discuss to kind of like guards to kind of like make sure that we don't do anything terribly wrong but we can kind of make this assumption try and make things fast and if they turn out not to be true then we'll have to go back to this low root. And sort of the other key piece here is like most programs spend a lot of their time in a relatively small number of places. So if you have like one loop where you're doing the majority of your computation or a bunch of loops where you're doing the majority of your computation, relatively speaking they're probably not like, you know your program probably doesn't spend most of its time in a full 50% of its source code. It's probably somewhere in a bunch of kind of localized places and so what this means combined with what we said before is basically if we can just kind of hit those places and compile them at runtime through the JIT we're gonna basically speed up your entire program because that's where your program is spending most of its time. So all that was kind of section one that was hopefully enough background information just to see what the general strategy is. I'm gonna transition to talking about stuff that we've learned along the way at Magnetic. We've been running PyPy in production at very large scale for probably about three years now. So I'm gonna talk about a bunch of stuff which shouldn't be terribly surprising but just some of the stuff that we've kind of had to deal with along the way and it's some of which I hope is fairly expected but any questions on sort of the first half of things? Okay, I'll be more time at the end. So just in a general sense it's probably most pertinent to talk about like our largest app which we run on let's say 100 servers does like 400K or 500K QPS and just from like literally taking the word Python and replacing it with PyPy just like magic 30% speed up without doing any work. And of course I mentioned before that PyPy is only gonna help you if you're a CPU bound or if you're doing a lot of computation this app isn't CPU bound which is why that number isn't way larger. So this is like a web app you can think of. So it has like a portion of what it's doing that's kind of like CPU intensive and that part of the request will end up getting sped up but overall it's like it's a web app. Not a client facing web app but it's a web app. And so 30% from an app that's basically spending a quarter of its request doing CPU bound stuff is just like literally money in the bank. Okay, so sort of the first thing you'll probably encounter assuming you're convinced so far. So if you do switch to PyPy the first thing you'll probably encounter is like, okay so are all of the libraries that I wanna use actually compatible? So it's actually a decent story for that now in the sense that like there's a website it's packages.pypy.org the URL isn't on there I apologize but it shouldn't be hard to Google if you don't remember packages.pypy.org. So what it is is basically a list of the top thousand packages on PyPy very confusingly not PyPy. So on PyPy, listing of the top thousand packages and then whether or not they're compatible with PyPy actually don't recommend you check it out because I can give you the summary right now. 99 or probably about 95% are and the ones that aren't are things in the kind of NumPy ecosystem other than NumPy itself which is undergoing porting. So like a large majority of the stuff that's on that list is gonna be of the 5% or however many percent is gonna be stuff in that ecosystem so we'll come back to in the end. Otherwise the large majority of things on that list are just green so I would say at this point in PyPy's life cycle just assume that things are going to be compatible other than a specific list and basically again the scientific stuff is certainly one section and certainly needs addressing and the other stuff is going to be libraries that use C extensions are gonna be either iffy or slow because the C API is C Python's C API and so PyPy emulates whatever parts of that it can but even the parts that can't, sorry there are parts of it that it can't so popular examples of libraries there if you're using MySQL Python which I would recommend well first of all they try not to use MySQL but even if you are don't use MySQL Python if you can it's quite old and even if you're using MySQL there are newer libraries out there but if you are it does have a C extension I don't think it will install on PyPy at all. Other libraries I think the Postgres library for Python Psycho PG2 I think also tries to compile a C extension so there are a couple of examples like that but generally speaking for those kind of libraries that have C extensions either it will not use the C extension on PyPy or there are kind of like equivalent libraries so like the Postgres one there's a Psycho PG2 CFFI library which is basically a drop in replacement but certainly like are all of the libraries I'm using compatible with PyPy is the first thing you'll probably have to check if you're porting over an existing thing and again the answer is basically if they're pure Python yes absolutely if they're not either look for pure Python replacements or it might work anyways because of the C Python C API but it's kind of a jarring transition to make when you're first using PyPy because what you start to look for is the exact opposite of what you looked for before because now every time you see a read me you probably want to check and look that it says pure Python which means faster on PyPy rather than what you were looking for before which certainly tends to happen in read me is where they call out hey I have a really fast C extension that I use to actually do computation that's going to make your program slower unfortunately. Other things that you'll probably encounter so this didn't happen to our own code but you'll find lots of programs with bugs in it unfortunately and bugs in a specific sense here so unfortunately this code has a bug and a bug other than the fact that I didn't define process here like this code leaks a file descriptor so it opens a file reads it and then never kind of like closes the file you won't notice this bug on C Python because C Python has a reference counting implementation but on implementations of Python including PyPy but not only PyPy that don't use reference counting this will leak a file descriptor which will maybe eventually get closed but if you're doing this in a big loop and you try opening a whole bunch of these like your program's gonna crash or sorry your program's gonna run out of file descriptors thankfully our own code didn't have I think any of these issues but you will run into if you try running we were on DevPy under PyPy and it had one of these issues recently so like other people's code that you run into might potentially have these sorts of issues and you kind of will have to know to look for them if you see a program run out of file descriptors your first guess when you're using PyPy is like okay someone forgot to close a bunch of files and is relying on kind of like C Python specific behavior there which again the same thing will happen if you run this code under like Iron Python or like Jython or any of the other implementations of Python that don't use reference counting kind of the next thing that you'll probably run into maybe the first thing for those of you who are like Ops see people in the room is handling provisioning, excuse me so if you go on the PyPy website they do have downloads you can like download PyPy binaries unfortunately if you're running like Linux the world is kind of sad when it comes to binaries in the sense that like the binaries that are available are kind of like dynamically linked to some system libraries so when you download a library from PyPy.org if you're on Linux you need to be on the distribution that it was actually compiled for otherwise it's unlikely to work they do have some like semi-portable versions but what we do is basically we use so that portable PyPy thing there which if you're on Linux and you want to deploy PyPy I would say just go straight here even if you are on one of those other distributions it's just easy we basically just plop portable PyPy everywhere and what it is is basically like it's a tar ball with PyPy inside and the PyPy in the tar ball is kind of scrubbed in a way that it actually will run kind of portably so we run CentOS and kind of the PyPy release cycle is fairly fast and generally speaking you do want to kind of stay as current as you can because they do make kind of very significant improvements as time goes on and so like he basically puts out portable PyPy tar balls like a day after the release is done so we kind of just have some provisioning scripts that basically will go off grab the newer version and just like kind of upgrade the whole infrastructure to whatever next version kind of completely portably at least for our sake. There are some issues this one I pulled kind of selfishly so certainly like again the world of Linux is rough when it comes to binaries so like you might notice things like this which is like it statically links against open SSL and like because it does that like there are kind of hard coded paths that it's expecting I think this is an open SSL problem more than a portable PyPy problem but like it won't be able to find your system certificate bundle which basically leads to things like this where like we don't actually do any SSL from within our server apps but if we did we probably would have noticed this sooner which is like it can't find any certificates which means you can't verify any certificates that you might be receiving and so you have to know to kind of like set an environment variable so that when you run that portable PyPy distribution it knows where to find the certificates on your system. So there are certainly issues I would probably not to apologize for them but I would probably call these more like Linux issues or open SSL issues more than PyPy issues but certainly you do need to be aware of things like this that it is kind of a story to download up-to-date binaries on Linux unless you're gonna build them yourself which you can do which I'd recommend you do at least once but it's kind of annoying to automate a little bit because building PyPy takes about an hour or an hour and a half. Okay so provisioning there's also a story for deployment so in the normal world you're potentially used to doing deployment kind of like this certainly this is how we used to do it so when you do a deployment you push out the new version of your code and then restart the service. Unfortunately if you do this with PyPy your entire infrastructure is just gonna like get distracted doing jitting instead of actually doing any of the work that your service usually does because if you run let's say a web app with a whole bunch of processes and then you restart them something that we didn't really talk about before is basically the jit kind of needs to run and do the compilation when it notices that your program is hot in a certain area and it tries to compile a particular area of your program so if you kind of like deployed your whole infrastructure and then just restart all your services for the first X number of seconds your entire infrastructure is gonna basically be compiling stuff instead of serving any requests so that's not that great so instead what you have to do is kind of like a more like rolling sort of thing so what we do is fairly simple literally right now what we do is we basically go off to all of our services all of them kind of pre-fork so we actually use G-Unicorn to serve web apps let's say so we have G-Unicorn pre-forking 24 child processes and then instead of doing a restart which would cause the whole machine to start compiling instead of doing anything useful we basically kill off each of the processes one by one with a delay in between so that kind of like lets the machine keep serving some requests while still actually restarting the process it's not foolproof certainly and it complicates deployments a little bit because if you do change something that needed to actually change in the master process too you need to know that you need to do a full restart otherwise when you fork you won't actually get whatever change that you wanted to so for example if you actually change the underlying version of PyPy that you were using and then you fork from the master process that you didn't kill you're gonna get the wrong binary you're gonna be running the old version this is kind of a funny problem that actually happened to us unfortunately and this is now complete marketing so I apologize PyPy was too fast in the sense that when we switched our last remaining app over to it it actually blew out or kind of saturated our firewall so we literally had to roll it back I think it's probably still rolled back right yeah so it's waiting for firewall upgrades so beware of your network it's a great problem to have I just wanted to put GDBM on a slide just because how many people in this room know what GDBM is? Is it more than one? None How come you guys aren't raising your hand we use it? GDBM is basically the sequel light of key value stores yeah it's a DBM so how many people now know that it's in the standard library so yeah so there's a standard library module implementing GDBM we use it for okay reasons PyPy didn't have the GDBM module when we actually switched to PyPy three years ago for quite fantastic reasons basically because the CPython didn't have any tests for it so they didn't know it existed so like PyPy runs the CPython test suite but CPython had no tests for it so the PyPy guys didn't notice it existed so they had to add it afterwards so I put this up here mostly to tell that story but also because those sorts of things do come up every once in a while where unfortunately like every piece of software CPython's test suite has a bunch of holes in a couple of places most of them not often traveled like GDBM and so every once in a while you run into something that like doesn't work on PyPy but maybe doesn't really work on CPython either because there aren't any tests actually verifying that does the right thing so every once in a while this does come up where you'll kind of hit a dark corner of PyPython and it won't work because there's no tests verifying that it works upstream either and then I would be remiss to not talk a little bit about profiling not very much about specifics so like if you're in the CPython world I'm sure you're fairly used to running let's say the C profile module or like time it or sort of like these standard library little profiling tools maybe you're used to other ones too we have to be a little bit careful when you're using PyPy how you interpret their results so certainly because of JIT warmup like we talked about before if you just run some sort of profiling without doing that your results are pretty much meaningless because you're measuring the portion of your program your timing things in your program before they actually have sort of reached their final characteristics but once they do reach those final characteristics the sort of like information that you're getting out of C profile is still misleading because it's still telling you about kind of like Python code that's executing instead of giving you information about potentially Jitted code that's executing and so the world of profiling is a little bit different for PyPy there's this thing called VMProf which is somewhat immature but definitely usable it will tell you things like what portions of your code have been Jitted already it'll tell you like your code has been 80% Jitted and it'll tell you to sort of like which functions have been Jitted like there are sort of like internal PyPy tools for telling you what the JIT is doing when it's compiling all those sorts of things so there's kind of a slightly either expanded or modified tool set for doing profiling which you do need to be aware of and then before I get to this question which is obviously the most subjective one in the whole talk there are just slipped my mind so never mind we'll just get right to this so why doesn't everyone use this if I remember we'll come back to it so the first reason which probably covers 90% of the answer to this question is like performance doesn't matter for most programs performance doesn't matter specifically if you're IO bound like we said before so PyPy is gonna help you again kind of get rid of that step where you're executing lots of instructions instead of executing some smaller number for most applications performance doesn't matter it's fast enough I like mentioning because it makes it kind of like beefs us up as like Python people there's a guy, Greg Wilson who likes to talk about the study done by someone whose name unfortunately I've forgotten where basically a bunch of people were asked to implement a particular algorithm in Python, in C, in Java and if you haven't read that study the results basically said like the ultimate source code that was produced at least between let's say Python and C differed by like 1.3 times so whereas the languages have very different performance characteristics when you ask humans to write programs in them they tend to usually write programs that are roughly the same speed even without all the stuff that we've talked about so it's kind of an interesting side study which I will try linking to the slides so even if performance doesn't matter occasionally performance doesn't matter yet because you have those other solutions that we talked about before and then once we get past those two we get into kind of more interesting answers for the remaining 10% so if you're deeply invested in C Python again the scientific areas of the ecosystem do tend to have issues moving over most of those are related to how deeply those that ecosystem is already invested in external languages so we mentioned you know PyPy is going to make your Python code fast and it's trying to discourage you from taking sections and writing it in some other language unfortunately because that ecosystem already wanted fast code large portions of that ecosystem are written in other languages Fortran, C, blah, blah, blah and then this one so there is PyPy for Python 3 I've never heard of anyone running it in production but I don't run generally in circles where I hear about people running Python 3 in production regardless so if you are running Python 3 in production maybe try it out otherwise for us let's make code faster was an easy choice but certainly this does come up as a reason why people don't run it so this is my personal guess as to why more people don't use it is marketing that's why I'm here I don't work with them but like you should run it our internal policy which you guys should know is basically like all new applications target PyPy directly there's no reason why not to use it for everything at least for us and that's it questions?