 Hey, what's going on everybody? My name is John Hammond and welcome back to another Python programming tutorial So in this series, we're going to be looking at the URL lib module And this module is pretty awesome because it lets us access the internet and do stuff with HTTP requests FTP requests and even local file stuff So I want to preface it though by noting the URL lib module is kind of low on Pythons, I guess priority chain of resources that can access the internet for you Python 3 has came in and made things really weird They've got URL lib 2 as another module and there's even URL lib 3. So these Updated modules have more functionality. You can of course still use URL lib But it's a little deprecated and I don't want to say obsolete but for simple stuff it's okay to use I would think and They actually even recommend the request package or the request module. That's an even better thing for accessing HTTP stuff In fact, that's what I use almost all the time and hopefully we'll get into that real soon But just so you know, it's a little low on Python's chain when we're actually using URL lib But okay, it's all about accessing the world wide web It takes in universal resource locators or URLs and that's how it works So let's take a look at this stuff. Oh, there is a warning here though When you're working with secure connections HTTPS or hypertext transfer protocol secure It doesn't attempt to validate the server certificate. So it doesn't actually You know make sure it's secure So you really have to trust the web page or the site Or just know what you're doing or use at your own risk just something to note. Okay When we actually get into the code and the documentation function stuff, the first thing I want to cover is URL open So this is real simple. It's pretty much the function You'll probably be using all the time it opens up a URL address and a web page and just lets you do things with it it opens up kind of like a file like object like a typical Python file object and It's got the functions to read read line read line stuff like that Even the file no and close functions that you'd expect with the Python file But it also has these interesting functions called info get code and get URL. Those are specific to this URL lib module Other stuff to note. This doesn't have to be an HTTP Setup it can also be a local file if you use that as its scheme identifier. I Can also your work with FTP. I believe and stuff like that So it's worth reading and learning a little bit more about it's got proper support for the iterator protocol So you can use the next function and work through it like an iterator I typically don't do that, but if you find a use for it cool, you do your thing man and All right, let's go through and actually look at some of these functions I'm gonna fire up idle and we'll import URL lib It is in Python standard libraries. They don't have to install anything and we'll run URL lib open So it takes a URL. It's other got other optional arguments like data and proxies. Let's just go to Google Okay, it takes a second to connect. That's just fine and it actually returns this object for us So let's bind this to a variable. I'll just call mine page and now we can actually work with it So like I said, you can see it has all of these Actual file handling functions ones that you expect to see in a typical file object in Python But also has some other stuff like info get code and get URL. So the most interesting thing I think is read and That'll pretty much get the entire web page for you all the source code or the content whatever is returned by the server and With Google you can see it's a bunch of JavaScript. There's probably some HTML in there somewhere Yeah, and maybe CSS you can see mixed in there and Google. It's a massive fluster cuck Charlie Foxtrot But all right, the other stuff is get URL and They'll return simply the URL that you have are at There's another variable URL that pretty much returns the same thing. I believe get URL is just a getter function for this variable So a lot of you might be asking WTF why would you ever use this but the documentation actually notes it pretty well? You could be redirected when you actually access this web page the server could redirect you or change your location to another address The URL open function will kind of handle this on its own But if you're the programmer and you actually want to know where you went You can use this get URL method to keep track of where you are redirected to next one is actually get code and That will simply return the HTTP response status code 200 mean success So it's a good thing. We're seeing that and of course there is a note of this in the documentation Now if you actually look at Let's see. What have we got left? We've got the info Function and that'll actually return an interesting thing because it gives us this HTTP lib object We can use this to actually access some of the headers and I think cookies even that have been sent Items it's essentially a dictionary. I'm probably not going to go into a whole lot of detail about this because I honestly never use it, but You've got all of these other variables and things that are coming along with the request and the response So you can see them as keys as keys of the dictionary or even values To the dictionary that each one corresponds with its own key as you can tell But yeah, you can go through this and I'll look through if any if any of these are actually of use to you Honestly, I haven't found too much for it because The request module handles this a whole lot better than I think URL lib does but another thing to note The data argument in the URL open function if you specify that with HTTP You can specify a post argument as if you were sending a post method through the internet The data has to be encoded though with a URL encode function And there's actually an example of that way way down at the bottom of the page You can take a variable and save it as a dictionary format or dictionary data Run the URL encode function to get that to the proper form And then you can pass that right in as the data variable in your URL open function. So Okay, I think that's everything I did want to cover I think URL lib URL open dot read is what you'll use the most You might need to of course reload the page or save that as a file called contents the first time you read it Because you're not gonna be able to read it again There is no seek function It can only open URLs for reading no seek operations are available So you might have some trouble Actually, we're treating the data again without having to have to access it or ask the server to give it to you one more time But things to note. I'm sure you'll be able to find things out and use the Use the module to the way you'd like very very easily. So thanks for watching everybody. I hope you enjoyed this I'll see you in the next tutorial