 In this video we are going to learn further details about how the list data type works. So let's go ahead and let's call the file shallow versus deep copies. So that is the topic of this video. So before we see how copying the data within the list works, we have to construct an example, of course. And we are going to use the same example as in the previous video. So let's go ahead and let's create an empty list, which we also call empty. Let's go ahead and also create a simple list, which we call simple. And putting the two numbers 40 and 50 in simple, I call the list simple because it is not nested. So we could also call it non-nested list, for example. And inside our memory diagram, what we have done so far is simply we created here maybe an empty list. And now in this diagram here, I'm not going to draw the data types and you will see in a bit why to save some space. So simply create an empty list and there is no element inside this list. And let's create a second list example. And let's also do it in the same size maybe. And this second list holds two numbers, number 40 as an object, like this 40. And a second integer, which is the number 50. And again, I'm not writing any types here. We call this list the simple list just like that. So this is what has happened so far in this example. So now let's go ahead and create a nested list by using the literal syntax, so the bracket syntax. The first element is going to be a reference to the empty list. Then let's put in there 10 as an integer, maybe 20.0 as a floating point number, or maybe to keep things simple a bit, let's simply put in the number 20. In the previous video, we saw that lists can actually contain heterogeneous data, so elements of different data types and we don't need this in this example here. And then in the previous video, we used the term 30 here, the word 30 as a string, but to keep things simple, let's simply put the number 30 as an integer in there here. And then to end the list, we simply put a reference to the simple list above. So what happened in memory? Well, in memory, what is going to happen is we create another list object, maybe just like that here. And the first reference in this list is going to be a reference to the empty list. The second will be a reference, let's maybe put it down here to number 10. The second will be the number 20, of course, an object. The third will be number 30 as an integer. And then the fifth reference at index 4 is going to be a reference to the simple list, just like that. And then of course, the list has a name in the global scope, we call it nested, and we have another reference right here. Okay, so let's do a little bit of repetition from the previous video. So in the previous video, we talked about indexing and slicing, but we only did so on the first level, so to say. But now we are going to grab further into the data structure here. So let's go ahead and let's first look at nested and we see it is what we would expect it to be. And let's assume the task is, for some reason, I want to access the number 50 here. So how could I access the number 50? So what we could do is we could use the indexing operator and because the inner list here is the last element in the nested list, we could simply use the index negative one, which gives us back a reference to the list 40, 50. And now I want to get the second or the last element in this list. So how could I access that? Well, I simply repeat the indexing operator by saying index one here again. This is how I could grab inside an inner list, right? So I'm not only grabbing inside the outer list here, but I'm grabbing inside the inner list here. And I'm basically referencing the number 50 here. Okay, so this is how we can do indexing in two dimensions. And an example for that, although this is not the example we see here, is we could model, for example, a matrix as a list of lists. And then we could use chained indexing to access individual elements in the matrix. So let's continue and look at the slicing because we are going to need slicing here for the example. So for slicing, what we are doing is we are also using the indexing operator. And then we have to provide a start index. Let's say we want to go from the beginning, so let's use index zero. We want to go to the last element in the list. So this list object here, the nested list has five elements. So not six, but five, five, because this is the first one. And the last list here will be only be regarded as one element inside the list. And this happens to have the index four, right? So if we have five elements in the outer list, the nested list, then the last element would be index four. And therefore, if you want to slice out all the elements, we would go until index five, which is one greater than the four, that is the last index for the last element. Because the first index, the left index is always included, and the right index is always excluded. So if I execute that, I get back, as we see, basically another list with the same elements. And now whenever we go from the first to the last element, what we could do is we learn also that in the last video, we could simply skip the indices. So this notation that you sometimes see is what we refer to as the full slice. So we are slicing all the elements in order. And so you could think of that as taking a slice with the step size of one. But the step size of one, we can just skip here because one is the default step size. And now the question is, what do we get back here? So do we get back the same list or a new list? And to answer that, let's take whatever we get back from the full slice here and store it in a variable that we call nested copy. So let's look at nested copy. Well, nested copy seems to be the same list as nested. However, we can easily find out that this is not the case. So how can we do that? Well, I can ask Python, hey Python, is the nested list the same as the nested copy list by using the is operator? The is operator we saw briefly in the first chapter is the so-called identity operator, the identity comparison. So in other words, the is operator asks the question or answers the question, is the left hand operand and the right hand operand the same object? So let's do that and we get back false. So obviously we must have two lists with the same content here. So that is what we conventionally would call a copy. So let's look at the memory diagram. What has happened when we took the full slice? So when we took the full slice, what happened was the following. We get a new list object shown as here this list object. And now the important thing to learn is, and I'm going to use another color for that, the references inside this list are created by taking the full slice from this list. In other words, the references, so the arrows that are in this list here, they are simply copied over. So in other words, the first element is simply a reference to the empty list. The empty list is not copied. We simply copied the reference. The second element is a reference to the number 10. So the number 10 that is important, it is not copied. There is only one object with the value 10 here, one integer object, but it has now two references. The same holds, of course, true for the 20 and the 30, and also for the last reference, which references the simple list here. And now this list here, this is what we call the nested copy. So now, and also let's put the reference here, of course. So now what we can say is, we can say that these two list objects here, this one, the original and the nested copy here, they basically share the state. That's what a programmer would say. They share the same data, they share the same state. So when can that be a problem? So let's continue with the example here. So let's continue and do the following. I will go ahead and I will use the nested copy which has the same elements as before. And now I'm also going to use the indexing operator and I'm indexing the last element, which happens to be the list here, the 40, 50. And now we could maybe let's do that as an exercise. We could check if this list is the same as the simple list above and now the answer is true. Okay, so this is how we know that in the memory diagram, this arrow here and this arrow here must lead to the same object, which is the simple list. That is what this tells us. So now let's continue and using the syntax here. And now let's go ahead and take the full slice again of the inner list. So maybe let's do it like this. So this references the inner list here. And if I take the full slice of the inner list, not a semicolon but a colon, then I get another full slice. But now I'm going to use the slice notation to do the following. I'm going to replace using the assignment statement the numbers 40 and 50 with the numbers, let's say 40, 41 and 42, just like that. Let's look at the nested copy and we see that the last element is now a list with the numbers 40, 41, 42. That is what we did here in the previous cell. And now comes the problem. This is what confuses many people but using the memory diagram, there is actually no reason to be confused. So if we now look at the nested list, the original nested list, we also see that the value here has changed and that may be confusing because the nested list up here was defined to end with the numbers 40 and 50. And now by changing something inside the nested copy list, we also see the same change via the nested variable. So why is that? So what we have done and I'm going to use yet another color is using the slicing operator or the full slice and we assign to that the numbers 40, 41, 42. What we effectively did is we went ahead, we followed the nested copy list here, then we followed the last element to get to the simple list and then we used the assignment statement to get rid of these two references here and replace them with the number 40. Then the number 41 comes right here and the number 42, just like here. Now the thing to understand is those numbers here are gone, they are garbage collected and because all the references go to the same list and there is only one simple list, of course, that means every change that is done to this inner list, to this simple list here, can be seen via many ways, right? If, for example, we go ahead in the trip to the lab here and we ask Python here, what is the simple list? It is also 40, 41, 42. So we are following the simple variable here, we see the numbers 40, 41, 42. If we go ahead and we look at the nested list here, then we also see the numbers 40, 41, 42 because now we are following this reference and then this reference here and we also see the numbers 40, 41, 42. If you go via the nested copy and we go via the last reference in here, we also see the same numbers, right? So this is what you need to be careful with and whenever we have a situation like that in memory, we call that a so-called shallow copy. So a shallow copy, for example, can be created by using the full slice notation. So a full slice is indeed making a copy, but it is not copying the data, it is only copying the references. Therefore, we call it a shallow copy, all right? So that is an important concept and a shallow copy means after we made the copy, we have at least two objects to share the same references to the same inner objects, okay? This is what here we saw with the simple list. Actually, we had three references here that all point to the same list that was now changed. And so that is something you need to be careful with. So now let's look at a different approach of copying which is more familiar with how you would probably think of copying without any programming knowledge. So let's do the following. I'm going to import the so-called deep copy module or the copy module. So let's import copy. So copy is a module that is in the standard library and copy, the copy module has a function defined which is called deep copy. And the deep copy is the opposite of a shallow copy. So let's go ahead and let's maybe go ahead and look at nested. It is just like that. And now let's create a deep copy of nested and that's assigned to that to a variable let's call it deep. So now let's look at deep. And deep happens to be a copy as we see from nested, right? So far everything is fine. But now let's do the following exercise. Let's go ahead and maybe first of all let's go ahead and see what has happened in memory. So what has happened in memory is the following. Python follows the nested list here because this is what is going to be copied and then it is going to create a new list object which mirrors this one here. However, the big difference now is the first reference which goes to an empty list. This empty list is also copied. So now we have an empty list here. Now here the number 10, 20 and 30 they are also copied. So we have a new 10 here. So let's put a 10 here, a 20 here and a 30 right here. And also the last reference which goes to this list here is also copied. So we get another reference here and another list and this other list up here now has the numbers 40, 41 and 42 in it. So first reference, second reference, third reference. And now we have a new variable that we call it deep so maybe let's put it right here, deep. So you see why I needed to save some space and the deep variable now points to this reference. So that is a deep copy here. So what is a deep copy in short? Well, a deep copy basically copies everything recursively. That is how the definition goes. So everything that can be reached in the nested variable is also copied. So now we have basically duplicated the entire data structure right here. To prove that this is indeed a copy let's go ahead and look at the last element in the deep list which happens to be 40, 41, 42 and let's assign to that to that inner list simply an empty list. And now let's look at deep and deep is now empty list 10, 20, 30 empty list. So in memory what we have done effectively is we follow the deep reference to the deep list here. We go inside the last reference and in here in this inner list here we now get rid of all the references. So this list here is now empty. Now let's go back into the trip to the lab here and let's look at the nested list and the nested list as we see still has the same 40, 41, 42 from before. So therefore in memory diagram what we see is if we follow the nested list and the last reference here we still point to the same list with the number 40, 41, 42 that was not changed. So the change in the last change you made was the change up here. So that is in memory diagram the difference between a shallow copy and a deep copy. A shallow copy was a copy where we only copy the references. This is what we did to create the nested copy and a deep copy is basically following some up to some object and then it looks at this object and it copies everything, every other object that can be reached via a reference from this list here. So this is why we have, why we duplicated the entire data structure. That is a deep copy. So that is an important concept often. So the reason why I introduce it in the course here is because I see that many students when they make copies they usually make shallow copies and then they make some changes to the shallow copy and then unfortunately the original data has also been changed and then they are confused and the reason for the confusion is simply because they do not understand the difference between a shallow and a deep copy. So oftentimes making a shallow copy is already good enough. So oftentimes having a second way to read the same data is often good enough. However if you copy data and you want to make changes and you want to preserve the original data then a shallow copy is not going to help you. You will need a deep copy. And the way to do that in Python is to use the deep copy function in the copy module in the standard library and also there are a couple of other ways of how to do that which I will talk about when it is appropriate in future videos. But for now note that whenever you get back a list for example by using the where is it by using the full slice right here then by default in Python you always get shallow copies. So most of the time you work with shallow copies and shallow copies are just good enough but sometimes in particular when you want to make changes to the data then maybe shallow copies are not good enough okay so maybe this concept here is a bit harder so maybe refuel the materials a bit rewatch the video until you understand it it's not super hard but it is still not too easy at the same time so I will see you in the next video where we talk about list methods and so what else can we do with lists basically so I will see you in the next video