 Hi, everyone. I'm Nina Zakarenko. I've been working as a software engineer professionally for well over a decade. I've worked at some companies that you might have heard of, like Meetup, Reddit, HBO. These days, I work at Microsoft as a cloud developer advocate. That means that my goal is to make Azure easier and better to use for Python developers everywhere. The slides for this talk are available online. So grab a copy to follow along or share with your coworkers. There are a lot of links throughout and a lot of helpful resources. I also tweeted a link to them if you want to grab them from there. My clicker is not working. I think Chris might have stolen mine or we got swapped. That's no big deal. OK, so let's try something. Let's see if this one works. OK, let me try one more thing. Technology. If it worked like we expected it to, we would all be out of a job. OK, let's try something fun during this talk. If you're learning something new, something you're excited about, you can go ahead and share a tweet using the hashtag NBPY. You can mention me. My username is ninja. That's like ninja, but without the I, NNJA. So let's dive in. Why should you care about memory management? Knowing about it helps you write more efficient code. The knowledge helps you solve issues with slow problems, slow programs that helps you troubleshoot, helps you debug. Sometimes you really want to answer the question, why does my program take up so much memory? Why does the memory usage increase over time? Today I'm going to give you the tools to help you answer that question. And as a bonus, writing more efficient programs makes you look smart. What are we going to learn today? You might have been interested in this topic before, but it's really easy to get overwhelmed. You're just faced with too much information, too much new terminology. So I'm going to teach you the vocabulary. Along with the basic concepts, the goal here is to make the topic understandable, even if you're just starting out. And this talk is going to give you the proper foundation for a deeper dive. What won't you learn today? Well, unfortunately, none of you are probably going to be experts at the end of this talk. That's the downside of the 1,000-foot overview. And that's OK. For those of you who are experts in the room, just a quick disclaimer that this talk applies to C-Python only. I heard a stiffled laugh somewhere. OK, so to really understand how memory management works in Python, we need to get a little philosophical here and ask ourselves, what's a variable? Here in this example, we have a C-style variable. It's a very simplified example. We have two variables, A and B. We're setting the value of both to the number 5. 5 is 101 in binary. We need to declare the type of the variable before assignment. We have to say int A equals 5. Why? Because those C-style values live in a fixed-sized bucket. It's a one-to-one mapping, a variable name, to a memory location, and a stored value. It's just a table. 101 is copied over and over to each memory location when we declare int equals 5. That's why you need to declare the type, so that the compiler knows how big to make the bucket. These buckets can only hold same-sized data, or an overflow error might occur. What happens when we change the value of our C-style variable? Let's say I now set the value of A to 6. When changing the value of that variable, the data in that memory location gets overwritten. So now the value is 110. I've went ahead and changed the value of A to 6 later on in my program. Python has names, not variables. How are Python objects stored in memory? With names, references, and objects. A name is just a label for an object. Each object can have lots of names, like A or B, X and Y, et cetera. These names reference that object. And think about two different categories of objects. We have simple objects, like numbers and strings. Simple objects are just that. They store their own value. Each unique simple object is generally only stored in memory once. Then we have container objects, things like dictionaries, lists, and user-defined classes. They're capable of storing references to simple objects or even other containers or container-type objects. What's a reference? A reference is a name or a container object that points to another object. And we're interested in keeping track of the number of references to a particular object. That's called the reference count. If you're on your phone or you're on your laptop, I recommend paying attention to this part, because it's the foundation of the talk. How do we increase the reference count? In this very simple example, we're setting X to be 300. That increases the reference count to the value 300 by 1. Know that in practice, for various reasons, the reference count to a particular object in a Python program doesn't always start at 0. Later on, we have our program. We say X equals 300. Now we're saying Y equals 300. Instead of creating a new slot in memory for that 300, like in our previous C-style example, we're just adding another reference to the existing object. Now we've increased the reference count to the value 300 by 2. In this example, we have Z. It's a list, one of those container objects. In our list, we're referencing 300 two more times. As you can see here, the references don't necessarily have to be explicitly named via a name. After we've declared X, Y, and Z, we've increased the reference count to the value 300 by 4 in total. Now that we've learned what increases the reference count, how do we decrease it? There's a few different ways. The first way we can do that is by changing the reference. In this example, I set X to true and Y to none. That means I've removed two references to the value 300. I've dropped that reference count to 300 by 2. Oh, I'm sorry. I see now that some of the arrows in my slides are not showing up. But there's some gray arrows between X and Y and 300 to signify that those references have been dropped. One more way of decreasing the reference count is with the Dell keyword. In this example, I've removed the reference from X to 300 by saying Dell X. That's decreased the reference count by 1. Note that this is just an example. In Python, you don't need to explicitly use the Dell statement for memory management. What exactly does the Dell statement do? Well, it doesn't delete objects. The name is a little bit confusing. Instead, what it does is it removes that name as a reference to that object. And that, in turn, reduces the reference count by 1. I have a really simple function here that I run. It's print hello. I set the greeting to equal the string hello. And then I print it out. As the function is running, the reference count to the word hello increases by 1. After that function is done running, our greeting variable goes out of scope. When that value goes out of scope, the reference count to it decreases. Once there are no more references, we don't really care if that object still exists. We can safely remove it from memory. And that's where we need to be careful with local versus global namespaces. If the ref count decreases when an object goes out of scope, what happens to objects in the global namespace? Well, they might never go out of scope, which means that their reference count might never be zero. So the trick here is to avoid putting any large or complex objects in the global namespace, because it might come back to bite you. Internally, every Python object holds three things. Type, a reference count, and its value. In our previous examples, we had two names, x and y. Those led to two references to one object in memory. This object knows its type, it knows that it's an integer, it knows that its reference count was two, and it knows that its value is 300. Let's prove that this is the case. Note that if you're trying this in the REPL, wrap your code in a function or try this out by running the code in a file, because the REPL uses optimizations that aren't present when an actual Python program is running. So the first thing that I do is I declare x and y, both equal to 300. And next, we're going to peek under the hood using the identity function. ID gives you the location and memory of a Python object. We'll see that when I print out the ID of x and the ID of y, the values are the same. You can also ask Python if these two names refer to the same object with the is keyword. That compares two objects to show us if they exist at the same memory location. So if I print out the value of x is y, I'll get true. This all leads into garbage collection. What is garbage collection? It's a way for a program to automatically release memory when the object taking up space is no longer in use. Back in the day, programmers had to allocate and deallocate memory manually. And spoiler alert, it kind of sucked. If you forgot to free memory, it could cause memory leaks. If you accidentally overwrote memory, your program could crash. Well, garbage collection to the rescue. I like to think of garbage collection being a little bit clearer when I think about it as memory recycling. There are two main types of garbage collection. The first is reference counting. The second is tracing. And in a way, Python uses both. So how does reference counting garbage collection work? As we saw before, we add and remove references. That rough count is increased on assignment. It's decreased when the reference is removed. When the rough count reaches zero, we can go ahead and remove the object. We know that we have no further use for it. And that could cause an interesting cascading effect because when you decrease the rough count of any objects that that deleted object was pointing to, if their rough count has now reached zero, you can delete them as well. That means that one rough count reaching zero can cause a lot of objects to be cleared from memory. Reference counting has, garbage collection, has some good parts to it. The first is that it's easy to implement. When the rough count reaches zero, those objects can be immediately deleted. But it also has some downsides. The bad. There's a lot of space overhead involved. You need to store a reference count for every object, and that data needs to live somewhere. There's also an execution overhead too, right? Because the reference count is changed on every assignment. And unfortunately, there's also an ugly side to reference counting. It's not generally thread safe. What happens if two threads try to increase and decrease the reference count of one object at the same time? We might end up with a pretty big problem. Reference counting garbage collection also doesn't detect cyclical references. What is a cyclical reference? Let's think about it by example. Here we have a simple node class. It contains a value, and it contains a reference to a child node. Let's say here we declare three nodes, a root node, a left node, and a right node. Then we point the root node at our left node, our left node at the right node, and our right node back to the left node. Our reference count of root is one. It's referenced to by its name. The reference count of left is three. It's referenced to by its name. It's referenced to by the root, and it's also referenced to by the right node. The reference count of right is two. It's referenced to by its name and by the left node. A cycle occurs when two objects point or refer to each other and nothing else. What happens when we remove the names as references to these nodes? We call del on root, left, and right. That means that the names root, left, and right don't refer to the nodes anymore, but the internal references, the ones stored in child are still there. Now the ref count of root is zero, but the ref count of left and right remain one because those nodes still refer to each other. At this point, those objects are no longer accessible from the program because those named references to them have been lost. Reference counting garbage collecting alone will not garbage collect objects with cyclical references. An early Python implementation was only reference count based, and it caused a lot of problems. Many types of objects can have cyclical references like graphs, doubly linked lists, and early on we realized that we needed something else. So there are two main types of garbage collection. We talked about reference counting. The second type is a strategy called tracing, and tracing garbage collection generally uses an algorithm called mark and sweep. It's run when the number of objects in memory is greater than a threshold. This first phase is called the marking phase. It involves starting at the root and traversing the reference graph, following all the references and marking any reachable ones. So in this example, the reachable objects are marked in green, but any unreachable objects remain blue. They're unmarked. When marking is complete, the sweep phase will then go and remove the unreachable objects, the ones that were unmarked. Cyclical references are caught with this algorithm. They're marked and swept, like the two nodes in the bottom left quarter. So what does Python use? We know that it uses reference counting, but it also uses a strategy called generational. Generational garbage collection is a type of tracing garbage collection. And we need another strategy because remember, reference counting doesn't clean up those dangling cyclical references. Generational garbage collection is based on the theory that most objects die young. How tragic. Frequently, objects are created to just store temporary values or for a function call, and they don't get used again. So how does this generational garbage collection work? Well, Python maintains a list of every object created as the program is run. Actually, it creates three lists, generation zero, one, and two. Newly created objects get put in generation zero. If an object survives garbage collection, it gets promoted to the next generation. I had to try very hard to not put a Star Trek joke in here. Keeping it professional. Every object is only stored in one generation, and we can optimize by collecting the young objects in generation zero more frequently than the old ones. Only container objects with a reference count greater than zero are stored in a generation list. It's similar to mark and sweep, but instead of holding a list of all the objects, it just tracks these, because they're the only types of objects where a cyclical reference could possibly occur. There are a few intricacies here between mutable and immutable containers that I won't get into, but overall it means that fewer objects are tracked and scanned, and it takes less time to run garbage collection. When the number of objects in a generation reaches the threshold, Python will run a garbage collection algorithm on that generation, and any generation is younger than it. So for collecting on generation two, that also means we're collecting on one and zero. So what happens during a generational garbage collection cycle? First, Python makes a list for all the objects to discard. It runs an algorithm to detect reference cycles that are out of scope for this talk. Sorry, but it's an interesting read, I highly recommend it. Then if an object doesn't have any outside references, it gets put on a discard list, and when that cycle is done, we can free up those objects, the ones on the discard list. After a garbage collection cycle, objects that survive that sweep will be promoted to the next generation, and that means that objects in the last generation, generation two, will stay there as the program executes. The big idea here, when an object's ref count reaches zero, we can remove it immediately. But if you have objects with cycles that need to be cleaned up, we have to wait for generational garbage collection to run. And in order to do that, we need to wait until the criteria for garbage collection has been reached. So if your program is slowing down or has increased memory usages, circular references could possibly be a culprit. And if we look back at our reference cycle from before, our node left and right, they retain their reference count of one, but these cyclical references will get cleaned up by our generational garbage collection. This is a question that I've seen quite a lot. People ask, why doesn't a Python program shrink in memory after a garbage collection? And there's a few factors here. The first is that the freed memory is fragmented. It's not necessarily freed in one continuous block. So when we say that memory is freed during garbage collection, what's happening is it's actually released back to Python to use for other objects and not necessarily released back to the system. Some quick optimizations that you might be able to make in your own code. One strategy that you can use to improve memory usage is something called slots, Dunder Slots, and Dunder stands for double underscore. We have to remember here that every Python instance contains a dictionary of its names and its values. I can make an empty dog class here, and I can set arbitrary values on the instance, in this case, for example, a name, buddy. If I print out the Dunder dict on my dog instance here, we'll see that it has a key, name, and a value, buddy. But what happens if we try to add a name to the string, hello? If I try to say hello.name is Fred. Well, we'll get an error that I'm sure most of us have seen before, an attribute error. Python is complaining at us. It's saying that the string object doesn't have an attribute name, and of course, that makes sense, right? We don't want to allow setting arbitrary attributes on our built-in types. We can use slots to emulate this behavior in our own classes. And prevent us from setting additional attributes on instances on the fly. Slots turn the internals of an object from a dictionary to a tuple. What's important about tuples? Tuples are immutable, meaning that their value can't be changed. So if I create a class point and I declare our Dunder slots to be x and y, now the only attributes that we can set are those that have been predefined. So I create a new point here. I can set x and y, but what happens if we try to assign a name to our point? We'll see one of those familiar attribute errors. And my point doesn't have that internal thunderedict. So I can't go ahead setting arbitrary values. This is important because if we use the getSizeOf function, that function returns the size in bytes for built-in data types. In my Python 3 implementation, the size of a dictionary in bytes is 232, but the size of a tuple is only 40 bytes. That's not a huge difference for one instance, but it can really add up when there are a lot of instances. So when might we want to use slots? If we're creating many instances of a class and if we know in advance what properties the class should have. There's a really great blog post on this topic from a company that was creating millions of instances of a particular class and they instantly dropped their memory usage by nine gigabytes by using slots. But an important aspect of their advice don't optimize prematurely and use this trick everywhere as it really affects code maintainability. Python also has a concept called weak references. A weak reference to an object is not enough to keep it alive. When the only remaining references are weak references, the object can be garbage collected. Weak references can be useful for a few scenarios, such as maintaining caches or maps to very large and expensive objects that take up a lot of room and memory. Now, let's quickly talk about the elephant in the room or maybe this shark, because the shark has gills. Okay, what is a gill? It's not something that we'll find on a fish. It's actually a global interpreter lock. The global interpreter lock prevents multiple Python threads from executing Python code at the same time. There's one gill for each interpreter. In other words, two Python programs running on one machine don't share a gill. Only one thread can run in the interpreter at a time. Why does Python need an interpreter lock? We needed to prevent reference counts from being changed concurrently because what if two threads were trying to increase and decrease the reference count of an object at the same time? Those operations might not happen in order, which might cause a big problem. What are the advantages and disadvantages of the gill? Well, the upside is that reference counting garbage collection is fast and it's easy to implement, but the downside is in a Python program, no matter how many threads exist, only one thread will be executed at a time. If you want to take advantage of multiple cores, use multiprocessing instead of multi-threading, each process will end up having its own gill and it'll be on the developer to figure out a way to share information between processes. The difference between a thread and a process is out of scope for this talk, but it's a topic I highly recommend reading about. So if the gill limits Python, can't we just remove it? Well, that's a great question, right? It's caused some infighting in the Python community and an early version of Python, version 1.5, a patch was submitted to remove the gill. Spoiler alert, it didn't go well. It sped up multi-threaded applications and that's a good thing, right? The downside was it slowed down single-threaded applications by about 50%. If this is a topic that you're very interested in, you can follow along with Larry Hastings' Gilectomy project. The project has stalled for various reasons, but you can catch his talks from Python 2016 and 2017. That means that for better or worse, the gill is here to stay. The Python codebase has become significantly more complex since that patch was introduced. So the gill is here to stay and core Python, but know that not all Python implementations have one. For example, Jython and IronPython don't have a gill. So, what did we learn today? We learned that garbage collection is pretty good. In most cases, it's better and less error-prone. With garbage collection, whole categories of bugs are eliminated or easier to avoid. And for most applications, it's easier and more straightforward than doing it manually in C, but it's not a magic wand. It adds a lot of overhead in terms of execution, time and space, and it still doesn't prevent us from being inefficient. Now that you know how memory is managed in Python, you're familiar with the concepts. I encourage you all to keep learning and exploring. It'll help you write better, faster, more efficient code. There's a lot I didn't cover today, so there is a bonus section with a lot of links and additional resources at the end of the slide. If you haven't switched yet, consider Python 3. It has a better, more efficient gill implementation. That's one of the many reasons to make the switch. And for scientific applications, use NumPy and Pandas. These packages are optimized to work with large numeric data structures, and they don't suffer from some of the same problems that we saw earlier. It's been a pleasure speaking with you today. If you want to learn more about Python at Microsoft, you can visit bit.ly slash nbpy-microsoft. You can follow me on Twitter and NJA to grab a link to the slides, and thank you all so much. So thank you, Nina, for that talk. What was it about? I don't remember. Me either. Oh. Something about memory. Oh, yes, right. Okay, thank you for talking to us about memory. Here is a wonderful design in Petaluma Camelback Water Bottle, with the North Bay Python logo on it. Thank you so much. Sam is trying to figure out which of the...