 Welcome back. I'm Dan. And I'm Nathan. And we're just here hanging out doing some coding with some pandas. They're gonna help us learn about pandas data frame. We're gonna pull data from all the different cities and then store them in a pandas data frame for easier access. So on our last video we left off with a price list of the first 20 results from a Craigslist query for the San Francisco Bay Area for the iPhone 5s. And that's what's shown here on the screen. Instead of dealing with lists for each city, though, we're just gonna go ahead and plop all that data right into a data frame. And we're gonna show you how to do that step by step here. First thing you're gonna want to do is import the pandas library. And we're gonna import that as PD. We also need the NumPy library. And we'll just call that NP. So to start off with, we're gonna start with an empty pandas data frame. And so we're gonna do this by doing df as our variable name equals PD dot data frames. This is how we can create just a data frame in various different ways. What we're doing here, though, is we're gonna create basically a large empty one just so the memory is all allocated. And it'll help speed up the process down the road when we're adding row after row after row. Our columns are just going to be city and price. These are because that's all the data we care about right now. And then the next thing is the index, we're going to set the index. This is what gives it the size. And we're going to set it to a numpy dot arrange, which is just a list ranging from the first value to the last value, doing 20 times the length of city dictionary will give us a good estimate of how many we're doing since we're limiting each of the cities to 20 results. We're just going to do the number of cities times 20, which turns out to be quite a lot. Down below, you'll see the code from our last module for the San Francisco area. We just copied it down here just so we have easy access to copy and paste the parts that we need. Because what we're going to be doing is using a lot of what we previously used. So if you haven't checked it out, please go back and check that video out. It'll be quite helpful. So since we're going across the various cities, we're going to need a for loop to go across that that list of cities. For this, we're just doing for city and city dictionary. That'll just go one by one through all the different cities. Once we once it we pull up a city from that city, Dick, what we need to do is create a URL for that one city because each URL is going to be different. We need to make a request. And then with that response, we get back from that request. We need to make a beautiful soup. So I'll just say make soup. Then then from there, we're going to pull all the prices off of that given website, which we're just going to limit to 20 of them. But with each of those prices, from there, we're just going to drop the data right into the panda's data frame or DF. We're going to skip the list just because that's just an extra step at this point, rather than making a list out of it and then making that list into a data frame, we might as well just drop it into our data frame that we've already created. Alright, so let's do this. So we need to create a URL, like I said, for each city. So URL is going to be equal to the string HTTP colon plus the city, the city URL, dicked, oops, city, dicked at with the index of city. So for the given, the given loop that we're going through, we'll pull the correct URL for the given city out of that dictionary. And then as we did before, we're adding appending the search slash SSS. So that's just what we did down down here. So I'll go ahead and get rid of that since we don't need it anymore. Next, we'll be creating requests, or actually, more accurately, a response object from from the requests library dot get. And we're going to call that URL. We'll do headers equals to headers. And lastly, we will pass our parameters. And that search parameters we previously did defined before. So if you haven't seen that, please jump back to our previous video and check that out. And then just dropping that right into a beautiful soups, the good old r.content in the HTML parser should be very familiar to you guys now. All right. So now what we have to do is now that we got our information from our given website, we got to pull out our 20 different results. So we're going to do this by just like before, down at the bottom of the screen, you can see for I comma a in enumerates might as well just copy that. Yep. We're going to do a find all for all the a's with a class of I gallery. And then to simplify this a little bit, we're going to do price equals and just copy in what we did before. And as we said, we're only going to do 20 of them. So this is not the most efficiently, but it lets it be more adjustable. If we wanted to do down the road. And okay, so that's all the stuff we've done before. Now it's time to actually add to the data frame. So we're going to use the loc method of a data frame object. And what that does is it locates the the row or based on the index you pass it. So we're going to pass it new new index value that new index value, we're going to have to go up above our city loop and create a new index. We could use I but the problem with using I here is our numerator is going to be reset for every different city. So we want an index that's continually indexing a new value from the start of the first city to the end of the last city without being reset. So we'll feed that a list of the city and the price. You have to give it two different values since our columns for data frames or two different columns. And then we will augment our index value by one each loop. And this all actually replaces our price list dot append. So we don't need that anymore. But since we want our prices and integers, we do want to keep the integer price section and just restore that as price. Okay, there we go. And that's all there is to it. This kind of got messed up. I'll go ahead and stick that right here where it belongs. That's the step where we're actually pushing the data into that pandas data frame just for clarity. So I guess we can go ahead and run this up. And there we go. It's done. It's done running. We have a data frame now. One thing I should mention those we did have to change how we were getting a value out of that dictionary rather than having this city in quotes, we had to take the since it's a variable, we had to take quotes out. So just a little typo there. Be aware of that. Because cities definitely are variable, not a string. But now we have a data frame. And we could just start taking a peek at it. And you can see that it's got data in there, which is kind of cool. So data, we can do a df.head, but just because you're the top five values by default, just get info about about our data frame, you can see that we've got 8051 records. Pretty crazy. We could just do a df describe to sort of do some summary statistics on it. On our next video, we'll be playing around a little bit more with pandas to show you some of the things you can do with manipulating the data frame. So we look forward to seeing you. Be sure to hit subscribe. We'll see you then. Bye bye.