 In this video, I'm going to summarize a couple of things we have seen throughout this course in the context of the iterator discussion. So let's go ahead and call the file simply iterator examples. So let's first of all create a list of numbers as we always use for example purposes. It's going to be the list 7, 11, 8, 5 and 3 and then 12, 2 and 6, 9, 10 and 1, 4. So that is a list of numbers. So in terms of memory consumption, this is going to create a list object and 12 integer objects and the list has 12 references to these 12 different integer objects. So that is what I would call a materialized object. So that is not a technical term that you will find in literature, but I use it often to make a parent a difference between something that does not occupy memory versus something that does. So a list object occupies memory. On the contrary, if I want to model the same numbers with a different order, let's say in the sorted order, what you could do instead is we could introduce, let's call it a variable, let's call it memory less because it should indicate that the following object has basically no memory and let's simply use a range object and let's go from 1 through 13, which is modeling the same numbers. So both numbers and memory less basically models the same numbers. Of course, in different order and sometimes order is important, but I am disregarding the order here. So my claim is that the range object is also something is a special kind of an iterator. So actually range is a very special object. There is also a link in the notes why range is super special, a super special case, but for now let's treat it as if it were an iterator. So let's open Python tutor and compare the two in terms of memory. So let's first create a numbers list and then second, let's create this variable called memory less, which is a range object. So let's go ahead and first create numbers and then second, we create memory less. So I call it memory less because no matter how much, how big numbers or how big of a sequence of numbers we are modeling, let's say this is not one, but let's say one billion and let's say this is one and this is 13 billion. So we have, let's say, modeling many, many different numbers. The range object in memory is not going to grow. The only thing that is different is the parameters that is stored on top of the range object. However, if I want to model, let's say, 10 billion numbers using a list object, all of them have to be in memory. So therefore a list object really grows in terms of memory consumption proportionally to the number of elements it has. The range object does not grow. So using the range object, we can model any sequence of numbers as long as of course we can express it as a sequence using a step size. You all know, you all remember how the range built in works, but let's assume we can model a series of numbers using the range built in, then using the range built in basically takes no memory because we don't have to store any numbers. So what is going to happen is we loop over both of them. So if I loop, for example, by saying four number in numbers, let's say print number and let's print that again on one line. What happens if I do that in memory? And also what happens if I do the very same thing here? But I'm going to use, instead of numbers, I'm going to use memory less. So if we go ahead and let's go ahead and put the for loop right here and let's put the seconds for loop down here, what we are going to see is simply that when we iterate over the list object, we obtain a reference to the numbers that are already there. So we don't have a second number here actually, but really we get back a reference to one of the numbers in every iteration of the for loop. And now we get to the end of the first for loop. And now comes the second object called memory less. And what happens in memory when I loop over it? Well, it pulls out the numbers on a one by one basis as we've seen it overwrites the number singular variable. So it numbers going to be one, two, three and so on. And we simply see how we are going to print that out. But the important idea is that the range object also produces the numbers on a one by one basis. Therefore, I think of it as a generator or kind of like a generator. It generates the numbers on a one by one basis. So a range object, whenever you can use a range object instead of a list with the actual numbers in it, well, you should prefer the range object because it models the same numbers. But at all points in time, it only has one number in memory simultaneously. And then it pulls out the next number and overwrites the space from the first number, from the previous number. So in terms of looping, both the list object and the range object work exactly the same. However, the range object does not need to have all the numbers in memory simultaneously. Okay, so let's look at a couple of further built-ins we have seen throughout this course. So let's review the topic of sorting. So you all remember in the discussion on list methods, the list, every list object has a method called sort, which could sort the element in place. So let's go ahead and do that. So I'm going to say numbers.sort. And remember that the list object is mutable. And this is indicated by the fact that we don't see a return value here. So I just executed the code cell, but we don't see anything below the cell. If we look at the numbers, we see now numbers is sorted, right? Okay, so this is one way of sorting a list. And this happens in place, so this does not create a second list object. So what are other ways of sorting? Well, remember we have in the Python documentation, let's go here under built-in functions, a function called sorted right here. And it says it takes any iterable and it's going to return a new list with that is sorted. Okay, so let's do that. Let's use the sorted function and let's pass to it numbers. And now we see that we have output below the code cell, indicating that we get back a new cell and a new list object in this situation with the same order because we previously sorted the list in place. But now we get back a second list object where the numbers are also sorted. If we wanted to reverse it, we could do so by passing a flag called reverse equals true. And we get back a list object with the numbers in reverse sort order. And now if you look at numbers, numbers is of course still in the previously sorted order because the sorted function does not touch the underlying numbers list. So what happens if I use the sorted built-in on the range object? So what if I use it on the memory less range object? Well, I get back a list object. Okay, and that is something that we should expect from reading the documentation. The documentation says return a new sorted list. So no matter what we pass it in, we always get back a new list object. So therefore, what this basically does here, this basically materializes the list. So this is equivalent of calling the list constructor and passing it memory less. This would result in the same outcome. Of course, if you want to reverse it, you can also do that right here using the reverse flag. And now we get back a new list where the numbers are in reverse sort order. Okay, however, note that memory less is still only a range object without, so it's still a rule that can generate the numbers one by one. So one thing that I want to mention that is so special about the range built in is that we have seen in previous videos that a generator will be exhausted after some time. So let's go ahead and compare that to generators. So comparison with generators. So let's first create a generator. Let's simply call it squares. And let's derive it with a generator expression for using the parentheses notation. And let's say n squared or n in numbers. And I mistyped squares, so it should be u first, of course. So if I look at squares, it is a generator object. And if I pull out, let's say using the next built-in function, one number, I get back the number one. And now if I pull out all the remaining numbers, all the remaining squares, for example, using the list constructor, what's going to happen is I get all the numbers except the first because the first number was already pulled out. And you remember that a generator cannot go back. It can only go forward. However, if we use the range built in, so let's say if I write memory less again here, and let's say I want to materialize the numbers in memory less, I get it back. If I want to do that a second time, it works. I can pull out the same numbers as often as I want. If, however, I go ahead and try to pull out one further number from the squares, I will get back an empty list. Why? Because the squares generator has already been exhausted. So that is a big difference between, or that is a thing that makes the range object so special. The range object is never exhausted. So whenever you loop over it, you loop over it from the beginning to the end. And then it's simply when it reaches the end, the loop stops. But when you loop over the same range object a second time, it starts at the beginning again. So the range object is kind of special, but no need to really care about that too much. That is just something I want to mention here. Okay, so what else is there? So now we saw sorting and we did a little comparison. So let's go ahead and also look at the built-in function called reversed. So reversed. Okay, so let's go to the documentation one more time. And let's look what the documentation says for reversed. It says the argument has to be a sequence. So it has to fulfill all the four properties that we want to remember for sequences. Especially the one that it has an order. Without an order, it doesn't make sense to do that. And also a sequence has to have an end. So there has to be a finite number of things. Otherwise, you couldn't reverse it. If it is an infinite stream of data, you couldn't reverse it. And then it says return a reverse iterator. So in previous videos, when we talked about the four behaviors of a sequence, the fourth property of them was the reversible property. And I said to you that whenever an object supports being passed to the reversed built-in, just like numbers, so for example, the numbers list, then the object is considered reversible. So ordered in easy terms. So what do we get back here when we use the reversed built-in? Well, we get back an iterator. And in earlier videos, when we used the reversed built-in, you could use it so the way we used it all the time was with a for loop. So I would say for number in reversed numbers, print number, this is what you saw before. So this is how we used reverse before in this course. But now we now understand after chapter eight that what an iterator is. So what in other words, what the reversed built-in gives us back is an iterator that is a rule in memory that simply goes in backward order. So just like in the previous video where I compared the iterators versus the iterables in general, we saw how I can get a list iterator using the iterable in function from a list. That is a rule that goes in forward fashion. Now I can simply go in backward fashion. So maybe let's compare the two. So it's called the one thing here forward iterator. Forward iterator can be constructed by using the iterable in and we pass it in iterable. So numbers, for example, and this here is going to be the reverse iterator. And now what we can do is we can go ahead and we can use the next function on the forward iterator and we get the first number. If I use the next function on the reverse iterator, I get the last number and so on. If I do it a second time, I get the number two. And if I do it a second time here, I get the number 11, right? So this is also something that is not so trivial, actually, to understand. So note how when I switch between this cell here and I execute that and this cell, what I'm really doing is I'm looping over the same numbers list in parallel, twice, okay? So I'm looping in forward and backward fashion at the same time. And this doesn't matter because the numbers list is not the thing that does the iteration. The thing that does the iteration is the iterator that manages the iteration and by using the built-in iter function, we get a forward iterator. By using the built-in, by using the reverse built-in, we are going the backward way. Okay, so that's it. But other than that, it's the same concept, okay? So now you know what reversed really does. So now you're not confused anymore to see the return value of simply calling the reverse built-in just like that because before chapter eight, this really didn't make sense what you're seeing here, but now it does. So this is just a rule that knows how to do something without having it done yet, okay? So that is basically, yeah, all of the things I want to summarize here. So this video is more of a summary probably that compares a couple of ideas that you have seen throughout this course and puts them together into a kind of a bigger picture, okay? So note how before this chapter, you always had to materialize all the data in your computer's memory that is okay to do when you're learning, when you're a beginner. But at some point when you go into data science and you want to work with big amounts of data that maybe not that hardly fit into your computer's memory, working memory, then you need to come up with some strategies to deal with that. And the way to do that is by first understanding the concept of an iterator, how different kinds of iterators, so understanding map, filter, generator, and a couple of others. And yeah, there are actually, maybe I can show you a couple of more iterators that are built in. So one other thing that we have seen before is the zipper. So let's say I have a list called numbers, right? And let's say I have a second list called names. And let's put in the list, the names, let's make it easy. Let's say Achim and Berthold again, and of course also Caesar. And let's say you want to loop over the numbers list and the names list simultaneously in parallel. Then we saw the zip build-in, right? So if you give the zip build-in numbers as the first argument and names as the second argument, what you get back is a zip object. So here it does not say iterator, but the zip object really is an iterator. So maybe let's call that or store that in a variable called zipper. And with the zipper object, we can also pass it to next. And this will give us back a tuple, as we see, a tuple of a number and a name, right? And if I do that a second time, I get back two in Berthold. If I do it a third time, I get back three in Caesar. And if I do it one more time, what will I see? Well, yes, of course, a stop iteration exception because the generator or the iterator is at its end. It is exhausted. Another example we have seen before in this course is the enumerate or enumerating intervals. That's how I would call it. So let's assume we have an iterable like names and I want to enumerate it. So enumerate basically means I want to give every element in this iterable an index number. How can I do that? Well, I'm going to use the enumerate build-in. And I simply give it names and enumerate gives me back, as we see, some object of type enumerate. So what is that? Well, let's call it enumerator and enumerator. If I pass it to the next function, I get back zero and Achim. Next time I get back one in Berthold and then two in Caesar. But if I want to mimic the same as above, what I should do is I should provide the enumerate build-in a start value of let's say start equals one. And then I get back one Achim, two Berthold, three Caesar, and then stop iteration exception. So what we learn from that is Python uses iterators all over the place. And oftentimes we use them without knowing. And if you want to use them explicitly, probably the most important one to know is the generator expression. And also it's, yeah, let's say it's closed cousin, the generator function, and which creates a generator object. And a generator object is probably the most flexible iterator that we have, but Python uses iterators all over the place in its language everywhere. So that is a summary video. So I hope now with all these examples, you understand the difference between an iterator and an iterable and iterator really prepares you to work with big amounts of data, with streaming data, with data, but that is infinite. Okay. And that is really what you want to do. If you want to find the next unicorn, the next super startup, then you must be ready to work with big data. So I will see you in the next video.