 And we're here coming to you live from the kitchen today because we're going to make a beautiful soup today. So last week we left off with the request library. So we showed you how to make a request using the Python requests library. Just down below here we had our response.content which is where we left off and as you can see it's quite a bit of what an alphabet soup that is. Yeah, it's just a mess. We need to add some flavor to this. So to get this content a little more organized we're going to start off with importing beautiful soup. We do this with not just import beautiful soup but we're going to use it from BS4 import beautiful soup because we don't need the whole BS4 package, the BS4 package has even more stuff inside of it. We're just importing the beautiful soup and we're going to do it as BS just because we don't like typing as much. So to start off with we're going to name this as a soup object. So start off with beautiful soup BS with inside of it we're going to do response.content because that's the HTML text that we're sending into it so that it can make it so much prettier. And we need to tell it what kind of content it is and which is using the HTML dot parser. Yeah, so now we have this soup object and we can kind of do different stuff to it. So if we want to, one thing we could do is just prettify that HTML that was so messy up top here. We can use the prettify method and come up with sort of a nice looking set of HTML where you can sort of see a little bit better what's going on. This will throw in all the nice tabs and return lines just to make everything more human readable. And as we can see here, this is the content that we want to actually take, which is all our cities as well as the URLs. All these cities are in an HTML tag called UL, which stands for unordered list. And each city in that unordered list is a list item. And so we're going to start there with parsing this HTML. A beautiful soup, we can just pull, use a find all and find all of a given tag. We could do this one of two ways with find all ULs or find all LIs. LIs tend to be pretty common websites. So we're going to just do the UL because we know all of our stuff is inside of one UL. So as Nathan was saying, we're going to use the find all method and the parentheses, we're going to specify the UL tag as what to look for. Run this and it'll give us all the ULs in the whole HTML page. So as you can see, this is quite lengthy. And if we do a length of this object, we'll see how many actual ULs we're in this. Oh, it's not just our one. One more than we wanted. We need to find out which one we want. Since we've already looked at this a little bit, we'll just show you what the last item in the list is since it's shorter. Oops, excuse me. So if we do a negative one here, we'll clearly see this is not the one we want. This is actually the footer of the whole page. But one thing I'll just point out here too is that you can see this class is a CL footer, whereas the class of the first element of the UL list is height six. So one way that we can create this city list item is to do a take that UL lists result set. And specify zero or instead we could do city list equals soup dot find all. And then we feed in addition to the tag that we're searching where we feed it a dictionary. And we say class. And the key is class and the value for this particular class was, was it header six? Is that what it was? Height six. Oh, excuse me. Height six. Height six. And so that'll give us the same thing. So there's a couple of different ways. There's many ways to skin a cat in this case. And so you can see that that's one way to do it. There are two ways to do it there. The one difference is to find all because it's finding all. If there's just in case there's multiples, you'll always get a list back. Hence why we added the bracket zero bracket because there was only one in this list. We only cared about the zero with one. If we, if we use the find method instead, it would just find the first instance of unordered list with this class, with this dictionary, with this class of height six specified. And in that case, it just gives you, we don't need the, it's not a result set. It's just the, the tag. We don't need the index. So moving on from here, we now have our list in HTML form of all our cities. Now we want to take out the actual HTML as well as not the HTML, the actual URLs for the cities as well as the city names to do this. We're going to do a for loop to go through this whole list. And by doing this, we're going to end up putting them in a dictionary also. So we're going to start off with a city dictionary that's empty. And then we're going to do a for loop. So we're going to do for city in city list dot find all is find all comes in super useful over and over. And we're doing an a here because all of our URLs are inside an a tag element. And that's the only thing that's in the a tag element. And so to fill in our dictionary, we're going to do city dictionary with city dot text because that's the actual text of the URL link, which happens to be our name in this case. And then we're going to equal it to city dot href href is rather than city dot city in brackets with href and quotes there. That finds up the attribute of the given tag. Exactly. So this will actually scroll through that whole unordered list, pulling out each single a tag and putting in the text as the key element of the dictionary and the URL as the value. So as you can see here, we have a huge list of cities. And URLs dictionary of dictionary of all these. Yeah. And as you'll notice, these aren't in order. So in Python 3.5 and earlier dictionaries are unordered. Apparently, that's going to be changing in Python 3.6. But for the moment, you'll notice that those are not in order. So you can't count on those being alphabetical. But for our cases, it's not really an issue as later on down the road. And once we have this in a database, we will be able to sort it then. So be sure to catch our next video about actually going to all these URLs and all right.