 Hello everyone, and welcome to our Selenium course. We're really excited to have you here. My name is Ben, and I'll be your instructor. I have over five years of Python experience. I've taught programming courses on Udemy in the past, and I've also taught them in the community. In my free time, I love to play guitar. So why should you take this course? First, you'll learn how to use Selenium for web testing automation. You'll also practice scraping data from real websites. And we'll use the page object model design pattern in order to structure our web testing. All of this will help you level up your Python skills and help you move forward in your career. Now, what will you get from this course? We have comprehensive video lectures on Selenium functionality with example code that you can download or follow along with. We have three guided projects to help you put your Selenium chops to work in the real world. And on the Udemy platform, you'll find a community of fellow students. And once you finish the course, you'll get a certificate of completion that you can post on your website or on your LinkedIn page. If all of that sounds good to you, we hope to see you in the course. Hello everyone and welcome to this video where we'll talk about some of the curriculum that we'll be going through in this Selenium course. We're going to start off with some of the basic elements of Selenium. And the first one is really locating page elements. So for example links or images on a web page. We'll learn how to do this for example using CSS or using XPath as a way of finding these elements. We'll talk about page navigation and also how to fill in forms with Selenium. We'll learn how to interact with text boxes, dropdowns, radio buttons, and all sorts of other web elements. Next, we'll talk about weights in Selenium. So there's two kinds explicit and implicit. And if you're not sure what those are, don't worry, we'll cover those in that section. And then we'll talk about the page object model, which is a Python design pattern specifically for Selenium. Finally, we're going to be using the page object model once we've learned about it for web testing. Our course is structured in a straightforward way. We'll have video lectures with slides that we'll go through. And then we'll head over to either VS Code or our Jupyter notebooks to go through example code for demonstrating the concepts or functionality that we've just talked about. There are also three guided projects throughout the course that will help cement your knowledge. In terms of the course requirements, we expect that you know some Python and that you have it installed already. We also expect you have an integrated development environment, something like Visual Studio Code, which is what I'll be using, but any other Python IDE will be fine too. We'll be using Jupyter notebooks for a fair amount of the course as well, and we'll be using Google Chrome as our browser for Selenium. So you should have those installed, although we'll go through some of this in our setup section. Hello everyone, and welcome to our lecture where we'll go through our Python environment and web driver setup. For this course, I'll be using Python 3.8.10, but other versions are probably okay to use. You can download Python if you haven't already from the link provided on this slide. Because we'll be using Selenium to interact with Google Chrome, you'll need to have Google Chrome installed. The version that I'll be using is Google Chrome 108.0, and this is an important note. You'll need to make sure that the Chrome driver that you download matches your Google Chrome version. If you're using a different version of Google Chrome, make sure you download the appropriate Chrome driver, not just the one that I have downloaded. In terms of Python package requirements, you should see the requirements.txt file on the GitHub page, the GitHub repo, rather, for this course. If you'd like the link for downloading some of these web drivers, we'll be using Google Chrome, which you can download on the link provided here. But if you'd like to go through the course using a different browser, you can download the appropriate web driver from the links provided here. In the course, I'll be using the settings.py file in the GitHub repo for a number of different things. The first one is that I use it to store the URLs that are used in this course so that you can just import them into your Jupyter notebooks. I'll also use it to store the path to the Chrome driver executable so that we don't have to write that out all the time. It also stores some other useful or shared information that we'll use throughout the course. And just a note, if you find any of the websites in the settings.py file or the websites that we use in this course are out of date when you take the course, or they don't look like what they look like in the videos, you can try using the Internet Archive or the Wayback Machine to find a version of the website that's close to what we're using. I'll show you how to do that in the following part of the setup video. So now we'll head over to our desktop to get our environment set up. All right, so here we are. We're going to get started with setting up our Python environment. And I'm going to be doing this on Windows, but it's pretty similar for the other operating systems too. So like Mac or if you're on a Linux operating system, the steps should be more or less the same. So first I'm going to go to my Documents folder. Then I'm going to make a directory for the Selenium course. So I'll call this Selenium Udemy. And then I'll go into that directory. Now, the next thing that you want to do is you want to set up a Python virtual environment for this course so that you're not having a whole bunch of problems with dependencies and all that. So you can run python-mvn and then call it whatever you want. I'm going to say dot Selenium vn. So that's going to go into that dot Selenium vn directory. So you just got to wait for that to get created. Okay, so now that's done. We've got our virtual environment to activate it on Windows. You have to call the activate script like this. So on Mac or on Linux, I think it's a source. And then you go to the bin slash activate folder directory in that virtual environment. So now we're inside this virtual environment. You can see that here with the dot Selenium vn. So the virtual environment is activated. So if you want to deactivate it, you just have to right deactivate and then we're out of it. So I'm going to activate it again. So scripts slash activate. So now we're back in here. So we want some things, one of which is Jupyter Notebook. We don't have that. So if I try to run that, it's not a recognized internal or external command. So that's okay. We'll get our requirements installed. So let me open up. Let's see, do I have VS code? I don't have VS code open. Let's open up VS code. I'm just going to set up a couple files in here. So let me open up a folder. And we'll go to the folder we just created. So that's in Selenium Udemy. So select that folder. All right. And so here we are. Basically, there's nothing in there. We've just got the virtual environment folder. So there's a few files I'm going to set up here. You can clone the git repository. I'm not going to do that for now. I'm just going to do this as a one-off. But if you'd like, you can also git clone our repository to get some of these files. So I've got requirements.txt, settings.py file. And then I also use an environment file. But this is optional, so you don't have to do this one. And I'll show you why in a second. So first things first, requirements. We need to install all of our packages. If you go to the GitHub for this course, there's a requirements.txt file. It has all of the Python requirements, the package requirements that you need for the course, along with those versions that you need. So we can just take a look at that. Control all, control copy. And then come back over here to the requirements. And I think most of this comes from Selenium and Jupyter Notebook. Or Jupyter, rather, because Jupyter has a lot of stuff that comes along with it. So then what we can do, we can do pip install-r and then requirements.txt. And that'll start to install all of our requirements. So for now, I'm going to fast forward ahead until this is finished. All right, so now we've got everything installed, all of the packages that we need for the course. We can check this out. Let's try to run Jupyter Notebook this time. And it should open up in our browser now. And then we should be good to go. And we can start setting up Jupyter Notebooks. So we're going to open up a Jupyter Notebook and work in there if we'd like. Now there's a couple more things to do with the setup. So the first one is this settings.py file. So all this is, is this just stores some constants that are useful, things like the URLs for the course. All the URLs are in there, as well as the path for the Chrome driver. So I'm going to copy this. We can copy that and put it in our settings.py file. And again, this is just so that I don't have to type out these URLs or you don't have to type out these URLs. You can just import the constant, right? Because the URL is not going to change. So then we can save that. The keys here is this environment variable. This is optional. I did this because I was programming on a Windows machine and a Linux machine. And I didn't want to have to keep changing the Chrome driver path. So if you're only using one machine, you can just say Chrome driver path is equal to whatever the directory is, right? Some directory path, right? You can do that if you're only working on one machine. So I'm going to keep this as is. And all this does is it gets the Chrome driver path variable from this configuration file. So this is Chrome driver path. And then you have some sort of string there for it. So we've got our Python environment set up. The next piece is you should download the Chrome driver. So make sure that you check to match the version. So for example, if you have Chrome version 107, download the 107 Chrome driver. I've got Chrome version 108. So I downloaded Chrome driver 108. So the way you can check that is you go to Chrome. Then you can go to your settings, right? This triple dot here. Check the settings about Chrome. And I'll tell you the version there. So I've got version 108. So that's the one that I'll have to download. Now in terms of the where the Chrome driver is, the Chrome driver path, I just stuck mine in a folder called web drivers. So if I click on this properties here, it should tell me the path for it, right? So then I can just copy this. Let's copy that. And then if I go back to my environment file, then it's what is this? Chrome driver, right? That's the name. The name of the executable is just Chrome driver. So I can save that. And then that'll be that's my environment variable, the one environment variable that I have. So we've got all the constants in our settings file. Let's go back over to the Jupyter notebook. Now you should be able to, from that settings file, so from settings, you can import, let's say the Chrome driver path, right? So now we've got that path variable. So if I print Chrome driver path, then it'll give me some sort of string, right? And that's where the Chrome driver is. And then I can use that when I need, when I need to use it with Selenium. And we can check. So from Selenium import web driver, right? Okay, we've got Selenium there. So we've got everything that we need. A few more things. If you have trouble importing the constants from that settings file, you can check. You might need to change your environment variables. So you can edit the system environment variables on Windows, just look for this, or search for environment variables. This will open up the system properties. In the advanced tab, you can go to environment variables. And you can try adding a Python path. So this is the path to your directory. You can add that variable to the user. If that doesn't work, you can try adding it to the system variables as well. So you can see I've got a Python path in both places. And that's why I'm able to do the imports here. Again, if you're on a Mac or a Linux, look up how to add environment variables there for your system. Okay, so that's one part. So you should be able to import all of those URLs. Now, the last thing I want to talk about is if you take this course and you find that the URLs don't look the way you expect them to. So let's say python.org, right? So if I go to python.org right now, let's open that up and just say python.org, right? And it takes me here and it looks a certain way, and maybe it looks the way that it does on the video. If you find that the website has changed maybe to such an extent that it no longer looks like what you're expecting it to look like or what it looks like in the videos. The other option is you can use something called the Wayback Machine on the Internet Archive. So if I go to python.org here and browse the history, it'll give me a calendar with the times that it's been scraped. So we're in December. So if you wanted to get a website or a version of the python website that looks like something I am working on in the course, you can go to one of these snapshots and it'll take you to a snapshot of the website, right? So let's just wait for this to load. Now the one thing is that the HTML will be slightly different, right? So the main thing is you'll just have this banner up here at the top that you'll have to work around. But this is just an option if you find that any of the websites we use have changed significantly between the time this course was recorded and the time that you're using it. You can always go to the Wayback Machine and try to find a version of the website that more closely resembles what we're using. But for now, that should be everything for getting set up for the Selenium course. So we've got our Jupyter Notebook working. We've got Chrome. We've got the Chrome driver downloaded and installed from the Chrome driver website. So we can get started with coding. All right, thanks for joining this lecture, everyone. And in the next section, we'll go through how to use Chrome DevTools as well as what Selenium is and how it works. See you soon. Hello, everyone, and welcome to our lecture on Selenium use cases where we'll talk about the things that you can use Selenium for. For an idea of how you can use Selenium for your work, we'll have three guided projects in this course. The first one will be a web scraper for Wikipedia data. The second one will be a form filler for a banned booking form on a live website. And finally, we'll write a Python unit test for testing a booking form using the page object model. To show you what these use cases look like, let's head over to our Jupyter Notebooks and VS Code to see some examples. All right, so we're going to go through some of the use cases for Selenium that we're going to cover in the guided projects in this course. Our first guided project is going to be a web scraper or we'll do some data scraping from Wikipedia. So I've got Wikipedia up in the browser here using Selenium already. Now what we're going to go through or go through and do is we can do this here. We're basically going to go through using Python, so I'm just going to run this code. Don't worry about what it's doing for now because we'll go through that in the guided project. But basically what we can do with this is collect a whole bunch of data from Wikipedia. So you can see here on this page you've got links for the Low Memorial Library, Columbia University, and that's the data that we've collected using Selenium, using the web scraper that we've built in Selenium. So that's one example of something that you can use Selenium for is that you can scrape information off of a web page, a web page like Wikipedia. Our second project is going to be a form filler, so we're going to use Selenium to fill out a form. You can see how this might be related to automating things like making reservations or bookings for... booking for flights or restaurants, whatever you want. So let me just get this Selenium driver up and running. I'm going to move this over to the side. And so we've got this GoldBugs website. This is a banned website. And we're going to use Selenium to fill out this booking form here where you need to fill in your first name, last name, email, subject, message, and all that. So what I'm going to do, let's just run through this. I'm going to run all these cells. Don't worry too much about, again, what's going on because we'll go through this in the project and keep going through that. And then we can submit the form just so that you can see here. I didn't touch this browser. We did this all through Selenium. So all of this stuff is filled out now. And then we can submit the form if we'd like in order to book the band. So that's the second use case, really, is for automating things on the web. And that's the second guided project that we'll be going through. The other use case, the last use case that we'll be going through, is to use it for testing, to automate web testing. So I've got VS Code up here, and I've got three files up. This is going to be our last guided project. It's basically using something called the page object model to test the form that we just filled out. So this is like taking that form filling to the next level. We're going to test that if we don't fill out the required fields on the form, the website will reject that submission. If we do fill out the required fields, then the website will take that submission from us. So those are going to be the tests that we're going to run. We have some other files that we'll fill out here. Again, in that guided project, so that'll make more sense once you get to it. And so let's just run this so that you can see what this use case looks like. We're basically running a unit test here on the form. So it opens up the websites, fills things out, encounters an error, then doesn't encounter an error, and then we've checked our tests have run successfully. So we know that our booking form on this website works the way we expect it to. So that's the third use case for Selenium that we're going to be covering in this course. If all of that sounds interesting to you, we hope you'll purchase the course. And if you've already purchased the course, we'll see you in the next section where we'll start to go through Selenium and how we use it. Thanks for joining, and we'll see you in the next lecture. Hello, everyone, and welcome to our introduction to Chrome DevTools. We'll be using Chrome DevTools a lot in this course, so it's important to understand what it is and how to use it. So what is Chrome DevTools? Chrome DevTools are a set of web developer tools built directly in the Google Chrome browser. That's from their documentation. Basically, it just makes your life a lot easier when you're trying to understand how web pages work. If you'd like to look at the documentation, we've provided a link for you on this slide. We'll go through a couple of the useful tabs that are good to know about, and then we'll head over to Chrome to show you what they look like in the real world. First off is the Elements tab. This is useful for inspecting the DOM and DOM elements when you're interacting with a web page. In terms of our Selenium use case for this tab, it's basically everything. We'll be using it throughout the course, from the start to the finish, in order to find where elements on the web page are and how to access them. The Console tab is also pretty useful. You can use it to view logged messages from the web page or to run JavaScript on your own. Our Selenium use case for this tab is mostly for verifying locators on our page, especially while using the XPath. If you don't know what that is, don't worry. We'll go through it in a later lecture. And finally, the Application tab. You can use this tab to view and edit storage variables or cookies on the web page. In terms of our Selenium use case, we can use this to verify stored variables or cookies if we're using those for our testing. Let's head over to our Chrome browser now to get a look at the Chrome DevTools application. All right, so we're going to take a look at Chrome DevTools. So the first step is to get your Chrome browser open. And I've gone to thegoldbugs.com for this section or this lecture so that we can take a look at this website. It's a pretty straightforward website. There are some sections. There's a booking form. And that's about it. You can go to really any website that you want to to get familiar with Chrome DevTools though. So now if I want to open up the Chrome DevTools console, I can right-click and then hit Inspect. And that will pop it open. And the tab I'll be open to is this Elements tab here, which is what gives us a DOM on the page. So I can click on any element on the page if I scroll down to the booking form or maybe on this button, please read our blog. What I can do is I can use this select element in the page to inspect it tool. And then I can click on that and it'll take me to that section of the DOM here. So I can see this button is actually an A tag. So it's an anchor tag. It'll take us to the slash blog section of the website. It has these classes on it and some other parameters and then this text inside of it. So some things to know about the Elements tab that's very useful. First you can double-click on things to select it. So if you want to copy the class here, right, or maybe you want to get an ID off of something so that you don't have to type this out, you can just double-click on it and then, you know, Ctrl-C to copy and paste that and use that in your code. So that's a good thing to know. The other good thing to know is that you can right-click on these DOM Elements and then you have a whole bunch of options. So you can add or edit attributes. That's more if you're trying to mess with the web page. What's more of interest to us is this copy section. So there's a few things that we're going to be interested in here. The main one is really the XPath and the full XPath. If you don't know what those are, those are just ways of referring to this element using something called XPath notation. So we'll be covering that in a later section, which is related to how you select Web Elements on a page. Or you could copy, you know, the styles or the selector. So that's the CSS selector, which could help us find this element on a page as well. So those are just some useful things to know about the Elements tab for the Chrome DevTools. So the console here, so this is where you'll get some errors if there are errors on the page. So it looks like there's a couple of errors on this page. But then you can also use it to run JavaScript if you need to. So down here at the bottom you could do something. Let's not that. You can just do console.log, for example. This is JavaScript for just printing something to the log. So hello world. And that'll just print there, and you can see that. So that's one way of using it if you need to test out some JavaScript for whatever reason. The other way that we'll use it sometimes is with $x, and that's for checking that XPath. So again, we'll go through the XPath and you'll learn more about it in that section. But just as an example, we'll give us all the anchors on the page. So now we'll get this list, and each one of those is some sort of anchor element on the page, and then we can get information about it. So that's the other use case for the console here, is that you can use it to test your XPath strings before you write them. And again, don't worry too much about how XPath works. We'll cover that in a later lecture. So sources, so this is just sources on the page. It'll have some files or whatnot. We're not going to mess around too much with this one. Network, we also won't mess around too much with. So there is, if you want to change the way the webpage loads, you can throttle here based on certain presets or custom settings for throttling. But we won't worry too much about the network now. We also won't worry too much about performance or memory. Those aren't very important to us. Application is one area where we will go into a bit. So this is where you'll have variables for local storage or variables for session storage that you can access through Selenium, as well as cookies on the page. So I don't think there's any... I guess there are a couple of cookies on this page. So you can see their names or values and other information about them. And we'll interact with this more when we get to that section on Selenium. And there are some other things here as well. But again, it's not really important to what we're doing with Selenium. Security tab, and then there's some other ones here. The other thing to go over is the dock side for the console. So I like to have it on the bottom. But if you want to have it on the side, you can have it on the right there. You can have it on the left. You can have it free-floating if you want. I usually like to have it on the bottom. And then there's some more tools. But the main things that we're going to be using for Selenium are really the elements and maybe the console sometime. But elements far more than anything. The last thing to be aware of is this device toolbar. So this is where you can emulate devices using Chrome tools. So we've got it on a Samsung Galaxy here. You can make it an iPad. And it'll show up with that ratio, that aspect. So this is, you know, 820 by 1180, 30%, because it's obviously not full-size. So if you want to emulate some sort of mobile device, you can use that with Chrome DevTools as well. And as always, if you want to learn more about Chrome DevTools and how they work, I'd advise you to go to the documentation because there's much more there than we're ever going to cover in this course. Again, like I said, the important things for our work with Selenium are really going to be navigating the DOM with the Elements tab as well as clicking on things on the page. And then occasionally maybe the console and, you know, devices or the application tab. But more than anything, it'll be the Elements tab. Okay, so that's a quick kind of rocket overview of Chrome DevTools and how it works and how you can use it. And you'll see more about how we use it with Selenium in later lectures. Thanks, everybody, for joining, and we'll see you in the next lecture. Hello, everyone, and welcome to our introduction to Selenium. In this lecture, we'll provide you a brief overview of what Selenium is... Hello, everyone, and welcome to our introduction to Selenium. In this lecture, we'll provide you with a brief overview of what Selenium is and how it works. So what is Selenium and what does it do? From the official Selenium website, Selenium automates browsers. That's it. That's all Selenium does. If you'd like to read more from the official site, the link is there on this slide. And if you'd like to read the Python documentation for the Python API, the link is provided here as well. What are some of the use cases for Selenium? You can use Selenium for web scraping or for data collection from websites. You can use it for taking screenshots or captures of the way a website looks or certain web elements on a website. You can also use Selenium for website testing and automation. And finally, you can use it for the automation of any sort of browser-based tasks. So why use Selenium and not just use APIs or HTTP requests, things like that? Basically, we need to use Selenium whenever we need to interact with a web page to get information or to check that something works as we intend it to work. Selenium enables browser automation via something called a web driver. The way this works is outlined in the diagram below. So for us, we have the language that we're going to use Selenium with. In our case, it's Python. You could use it with Java or something else, but we'll be using Python. Next, you'll have the API or the package that will help you interact with Selenium. So we have our Selenium Python package. Next, we'll have some sort of web driver. The web driver is basically the interface between your Selenium API and the browser. If you want to work with a Firefox browser, for example, your web driver will be something called Gecko driver. That's the web driver that allows you to interact with a Firefox browser. In this course, however, we'll be using Chrome, so our web driver will be the Chrome driver. That will be our interface between Python and Selenium and the Chrome driver, or the Chrome browser, rather. To use Selenium with Python, we need to have the following install. Python, of course, will need a Selenium package, so this is Python-Selenium. We'll need a browser-specific web driver, like we saw in the previous slide, and then we'll need that browser, the browser that matches the web driver we want to use. And just a note, the browser software and version should match the web driver that you're using. So if you have Chrome version 104, for example, you won't be able to use Chrome driver version 105. You'll either have to upgrade Chrome to version 105 or find a version 104 of the Chrome driver. To start your Selenium session in Python, we'll have to do a couple steps. It looks like a lot, but it will be pretty straightforward and you'll get used to doing it pretty quickly. We'll have to import the web driver, import a service for the specific browser we'll be using, instantiate a service object using a file path to the web driver, instantiate the web driver using the browser or service that we've just instantiated. And then once that's up, you can get the URL that you want to go to. Finally, whenever you're done using your web driver, you should always quit your web driver. What this looks like in Python code is shown here. So from Selenium, you'll import web driver. From selenium.webdriver.chrome.service, you'll import the service class. And then you can set up your driver. So you make your service object with the path to your web driver executable then you set up your driver and you'll just pass the service as the service parameter and then you can get your URL of interest. Once you're done working with that URL, you can quit the driver. Let's head over to a Jupyter Notebook now to see how this works. Alright, we've got our Jupyter Notebook open, so this is where we're going to start with learning just how you open up a Selenium web driver session. The first thing you need to do is you need to run through your imports. So that's what we're going to do. So from Selenium, we need the web driver. So let's import web driver. And then we'll also need the service class. So from selenium.webdriver.chrome.service, let's import service. Alright, so we'll run those imports. From our settings file, we'll import some utility constants. So that's our Chrome driver path, which is the path to the directory where we're storing our Chrome driver executable. And then I'm going to get the URL for python.org, which is really all it is. You can use a string if you want, but I'm just going to do that import. Next, to start the Chrome web driver, you first have to do a service. First have to start up that service. It used to be different where you could just do webdriver.chrome and then pass the executable path. But now you have to do it this way. So service, we want the executable path. That's going to be the path to our Chrome driver executable. Instantiate that. And now we can instantiate our driver. So all we have to do is webdriver.chrome and then we've got a service parameter where we will pass that service object. And now when we hit shift enter to run that cell, we'll get a Chrome web driver session that pops up here. So you can see this banner that tells us Chrome is being controlled by automated test software. That's our Selenium, right? That's our Python that we're writing here. Now if you want to get a URL, all you have to do is driver.get. We're going to get that Python URL. So I'll run that. It'll open up in the browser here. So it's taking us to python.org, right? And now if I wanted to, I could do whatever other stuff I needed to do in Selenium. So I could run a search. I could test some buttons. I could check things, verify that they're working as expected. Anything that I need to do in Selenium once I've gotten that URL via driver.get. And as always, a good thing to do or to remember to have at the end of all your scripts or your Jupyter notebooks is driver.quit so that you can quit that web driver session. And so you don't have that Chrome browser just hanging out there. So if I go down to this tab here now, you'll see the Chrome isn't there anymore because I've quit out of it. So that's the basics of how you set up a Selenium session. We'll be using that a lot so you'll get very familiar with it. But this is the bare bones of how you get a Selenium session up and going. Thanks everybody for joining. And we'll see you in the next lecture. Hello everyone. And welcome to our first lecture in the section on locating page elements in Selenium. In this lecture, we'll be talking about locating page elements using CSS selectors. In Selenium, we can use a CSS syntax or CSS identifiers to find elements in the web page. Some of the important parts to know are ones that you may have already seen if you've written CSS in the past. This includes HTML tags. So things like A for anchor, div for div, h1 for header one, so on and so forth. It includes class names. And you can specify this using the period notation or the dot notation. So if we wanted to specify a div with a class name of my div class, we could write div dot my div class in order to find that element in Selenium. And finally, we can use IDs as well. And these are specified with the hashtag or pound sign symbol. So if you wanted to find an anchor with an ID of myAID, you could type a pound sign myAID to find that element. As I mentioned, if you've written CSS before, this should look familiar to you. But it's common in other areas as well. So you can see it in emit shortcuts or emit cheat codes for building HTML pages in editors like VS Code. From a broader view of things, all the ways of locating page elements with Selenium really depend on two parts. So first, we'll want to know how we want to find the page element. That's the by part of locating the page element. And then we'll have a string of interest. So what's the string that tells us what to find? In terms of locating page elements with CSS selectors, for example, our by would be a CSS selector, and the string would be whatever CSS selector is of interest. So if we wanted to find all divs, we could write div, pass a string of div, or if we wanted to find something with a class name or an ID, again, we could use that syntax from the previous slide as our string. In terms of the methods of interest to us, there are two web driver methods in order to find the element or elements on the page. So first we have find element, which will find one or the first element if there's multiple elements satisfying those locator conditions. And then we've also got find elements with an S plural, so that will find all of the elements that satisfy those locator conditions. Let's look at some code examples to see how this works before we go into our Jupyter notebooks. So to start with, we have to have a new import, which is that from selenium.webdriver.com and .bi, where we import the by class. And the by class holds all the constants, really, or, yeah, all the constants in order to specify what way we're using to look for page elements. The other imports should be familiar to you from previous lectures. Once we have our imports, we'll set up our driver as before. And then we can get the URL of interest, so driver.get, and then you pass the URL. And then we can find elements by a CSS selector. So the examples are highlighted here in bold. So if we wanted to find an element on the page, we could write driver.findElement. Then the first parameter would be by.cssSelector, so that's the way that we're looking for things. And then we'll pass a string with some sort of CSS selector. And if we wanted to find all the elements satisfying or matching that CSS selector string, then we would use findElements with an s, plural again, but with basically the same parameters. So we'd still say by.cssSelector and then pass some sort of CSS selector string. Now that we've talked about how we locate page elements using CSS selectors and what those CSS selectors are, let's head over to our Jupyter notebooks and get some practice with this functionality. Okay, so here we are in our Jupyter notebook and we're going to start off with our imports. So from Selenium, we need to import the web driver. Then from, we need to import the Chrome service, so from selenium.webdriver.chrome.service. Let's import the service class. And then finally, like we saw in the lecture, we need to import that by class. So that's in webdriver.common.by and we'll import the by class. So that's everything we need from selenium. From our settings file, we can import the Chrome driver path and then we're going to look at the books to scrape URL. So this is a practice website. Finally, we'll set up our service and our web driver. So we have to instantiate the service object. So let's do executable path. Is the Chrome driver path, set that up and then set up the web driver.webdriver.chrome and the service parameter is the service we just set up. Then we'll wait and we should get our Chrome browser pop up here. Let me just move this over so that we can see the code as well. And then let's get the URL that we've got that we're going to take a look at. So driver.git, books to scrape URL and it takes us to our website. Okay, so let's take a look at how we find page elements with CSS selectors. So the first thing we're going to take a look at is this alert here, this warning or this div. So we can open up Chrome tools. Let's pin this to the bottom and then select that div and let's go to the elements and pull this down so that we can read it. And so we've got this div here. So what we're looking at for this warning is we've got a div with a class of alert and alert morning. So if we want to find that in Selenium, what we can do, let's call it element one, is we can do driver.findElement and what we want to search by is the CSS selector and the string of interest is going to be div.alert.alert warning. So we do that, we've selected that element. Let's verify that by taking a look at the text attribute. And so we've got warning, this is a demo website, blah, blah, blah, which is what it says here. So that's great, we've got that element selected. Now let's take a look at another element. Let's try this again with maybe something different. Let's take a look at this unordered list here. So let's click on this A. So you see we've got this unordered list and it's a nav list. So if we click on, or if we click on this in the inspector, you can see we've got that whole list of book categories. And if we want to select that, we can select this unordered list class nav nav list. So let's do that, we'll call that element two, driver dot find element. We're searching by CSS selector again. And this is going to be, let's make this on a new line, UL, so unordered list dot nav dot nav list, because those are the two classes. So now we've got it selected. If we want to take a look at the text, there's no text there. Or there is text, but it's that list of things. So it's got the new line. It's not formatted very nicely. So let's delete this, let's delete that cell just so that it's easier to read. And so we've taken a look at how to select things with CSS selectors using that class dot notation. But let's try looking for something with an ID. So what we're going to look for, let's do a control F, let's find my string. There's something in here, there's a div messages. Or let's look for messages. There's this div ID here. So there's a couple of divs under the page header. So we've got messages and we've got promotions. So we can select one of those using that ID syntax. So driver dot find element by CSS selector. And then we want div pound sign messages. If we take a look at the text, it's an empty div right now, so there should be no text. So you see we've got an empty string. And then finally, the last thing we're going to take a look at, since we can do this with all HTML tags, let's take a look at the images. So we're going to click on this. So you see we've got an image here and it's got an alt text. So let's select that image or select all the images rather. And we can use find elements plural for that. So image elements equals driver dot find elements. We're doing it by CSS selector again. And we want all the images. So we're just going to search by the tag with no further selectors. Okay, so we've got a list of things there. So let's print stuff out and see what we can get. So for element in image elements, write this for loop. Let's just print the element dot get attribute, which is a method to get attributes off the HTML object. And we'll get that alt text. So what we want if you look here, we've got the image and it's got this alt text attribute or a property in the HTML. So let's print that out. And you see we've got a list of all the books or the alt text on all the images. So all the books on the page. The light in the attic, tipping the velvet, so on and so forth. As always, we should remember to quit our driver. Once we're done with whatever we're doing in Selenium, so driver dot quit. And then we're back here in our code. Okay, so that was a lecture on how you select things, select page elements using CSS selectors. Thanks for joining and we'll see you in the next lecture. Hello everyone and welcome to our lecture on locating page elements using IDs. Some elements on any given HTML page will have an ID attribute associated with their HTML tag. We mostly see this with the important elements on a given page. For example, if you have a search box, the search box might have an ID of search. If you have a submit button for that search element, the submit button might have an ID of submit as well. An ID on any given element is unique within the HTML page. So it's a very useful tool if you want to select specific elements using Selenium. In terms of how you use this functionality to select elements in Selenium by their IDs, we start off with all of our classic imports. So the web driver, the service, and the by class. Then we have to set up our service and our driver like we've seen before. And next we'll get the URL that is of interest to us. To find an element by our ID, we'll use a similar method to what we saw with the CSS selector lecture. So we can use driver.findElement by.id and then we can pass the ID string. So in the example I mentioned before, if we have a search box with an ID of search, we can just say by.id and the string would be search. We can also use findElements to find things by ID, but this is probably not used very much in practice because as I mentioned, the ID on any element is going to be unique, so there won't be more than one thing that you can find using this method. Now that we've talked about how to find elements by their IDs, let's head over to our Jupyter Notebook to get some practice. So here we are in our Jupyter Notebooks and we've started off with the basic imports, so the web driver, the service, the by class, and then for this lecture from our settings file we'll import the Chrome driver path and the URL to the Selenium website. So that's the website we'll be using to look at things in this lecture. The service, we set up the service and the driver, we set up the driver, so we've got this going over here, full screen, and we'll switch over to it in a minute. Now the website that we're going to get is that Selenium website, so let's do driver.get Selenium URL and that'll take us to the Selenium website. So let's take a look at this page for a minute. We're going to be looking for things by ID. So if we inspect this search bar, actually, you can see it's actually in a div with an ID of docsearch. So that's good for us, that gives us something to look for. So if we go back to the Jupyter Notebook, let's get that element based on the ID. Take off the caps lock, so element one is driver.findElement by.id and then docsearch. And that's the ID like we can see in our Chrome DevTools. Okay, so we've got that element. Now let's take a look at it. One of the ways we can look at it is we can use the tag name attribute. So that'll give us the HTML tag name associated with the element. And here you can see it's a div. So that's what we expect because it's a div with an ID of docsearch. We can also get the inner HTML as well if we want. So we'll get the attribute inner HTML. And this is just so that we can take a look at some of the other methods associated with these web elements. Okay, so the inner HTML is everything that's inside this div. So you can see we've got that button. We've got a span, the SVG, all of that good stuff here that we can access with git attribute. Now there's also this logo element on the page. If I inspect that, you can see we've got this SVG object or tag, rather, and it's got an ID of selenium logo. So that's another thing that we can select using an ID in selenium. So we'll call this element 2. We'll use driver.findElement again. We're looking by ID and we can just get the selenium logo since that's the ID of the element. Okay, so we've got that. So we can check out the tag name again. It should be SVG. That's what we expect. And then we can use that git attribute method to get basically anything on this tag. So you can see we've got data name, XML, NS, ViewBox, all this stuff. So let's take a look at the data name, see if we can pull that out. So go back to Jupyter Notebook. We want the attribute of data name, and it says selenium logo, which is what's on there. And so you can select things, a lot of things on the page using different types of methods. So we talked about CSS selectors before, and in this lecture we talked about IDs, but you can combine them with the CSS selector method or by looking for things with CSS selectors. So here let's try to select that logo, but we'll use CSS selectors instead of IDs. So element three, let's just make this driver.find element. We're going to do this by CSS selector. And then what we want is the SVG pound sign or hashtag selenium logo. So that's the CSS selector way of specifying both the tag and the ID. So now we've got that. Let's take a look at it, get that attribute of data name again to verify that we've got the same object, same element on the page. And now we see we've got the selenium logo on there as well. So what you can see is that ID and CSS selector, as you can use one or the other, define the same element on the page depending on what you're looking for. And as always, make sure to quit your driver after you're done using it. Thanks everybody for joining this lecture where we saw how to select items in selenium using their IDs, and we'll see you in the next lecture. Hello everyone and welcome to our lecture on locating page elements using the name attribute. Within a given HTML page, some page elements will have a name attribute associated with their HTML tag. You mostly see this with things like buttons, forms, form elements, and things like that. Unlike the ID, the name attribute is not guaranteed to be unique within the HTML page. In terms of practical use, the name attribute is mostly relevant for passing HTTP requests to the server. So you might use it more from that perspective if you're using a package like Django where you're making those types of server requests or API calls. For our intents and purposes, however, with selenium, the name attribute really just provides us with another way of selecting data from a web page. To select a page element using the name attribute, you can import all of the normal stuff that we need to use selenium. So our web driver, our service, and our by class. Then we'll need to set up the service and the web driver like before and get the URL of interest, of course. And in order to find elements by the name, we can use driver.findElement as we've seen before. And then the first parameter will be by.name and we pass the name string in the second field. So if we're looking for something that has a name of search, for example, we could say by.name and then pass the string of search. As I mentioned, the name attribute is not necessarily unique on a page, so there may be more than one element. If we want to find all elements, we can use driver.findElements and the first parameter again will be by.name and then we can pass that name string in. Okay, let's head over to Jupyter Notebooks and get some practice using this functionality. All right, so here we are in our Jupyter Notebook and we're going to take a look at how you can locate page elements in selenium using their name. We start off with the imports as usual web driver service by and then from our settings file, we'll import the Chrome driver path and the URL for the Python website. So that's python.org basically. Then we'll start the service, start the driver, and we'll be good to go. So let's get that website now so we can use driver.get and we'll get that Python URL and that'll show up in our Chrome browser here. So now we've got it up. One of the things we might want to find if we look at things that have names is you can expect this and this actually has a name. So we've got this input that has an ID, so it's got ID search field that'd be one way to select it in selenium would be by ID but it's also got a name and the name is Q and so if you were working on back end systems that would be the variable name that gets passed to the server but we're not really interested in that with selenium we're just interested in grabbing the input. So we'll call it element one and we'll get it with driver.findElementBy then we'll use name and the name as we saw is Q. Not Q space, just Q. So hit that, shift enter and we've got the element there. So now if we look at the tag name like we were looking at in the previous lecture we can see that it's going to be an input. So run that, yeah it's an input type of HTML tag and then if we want to get the, maybe we want to get the class to let's look at the element one.getAttribute so we're using that getAttribute method again and we're going to get the class attribute. So you can see it's got a class of search field. If we look at that here, that's what the class is called in the source, the HTML source file. Okay, so that's that and maybe we want to take a look at the submit button as well because we look at the HTML here, we've got a button, it's a type submit and the name has submit as well and the ID is submit. So again you could search for this with ID or with by name but if you search by name it's not guaranteed to be unique, right? So you could select only this with ID or anything with the name of submit using the name. But let's head back to Jupyter Notebook now and try to select this object just with its name. So we'll call it element two, it's equal driver dot find element by name and we'll call it submit since that's the name of the element. Okay, run that, ran fine, element two dot text, see what the text is, go. So if you look at that, that's what's inside here, that's what's inside the button, right? So that's what we're selecting with the text attribute and the other thing we can select maybe is we can get the value. So element two dot get attribute and we'll get the attribute of value which has nothing in it right now because it's not filled out in that HTML, in the HTML of the web page. And just so that you know, you can also use these types of methods to get things out of the header of the HTML page. So there's typically a body and then a head object. So if I look at this here, we've got all these meta tags, right? With names associated with them. So you can actually use Selenium to pull this information or these meta tags if you'd like to do so. I don't know exactly what it'd be useful for, but it's something that's there if you want to get it. So we can do this, let's see, what are we going to use this with? We're going to find the keywords. So we can do, let's call it element three is equal to driver dot find element and then we're going to do by name and we're going to find keywords. And the keywords, let's try to find that in here. Here we've got the, okay, so here we've got the meta tag that we're going to be looking for and it's just basically a list of keywords associated with the web page. So let's run this. So we've got that element. So if we get the element three dot tag name, we'll see it's a meta tag, right? And then let's get maybe the content out of there. So get attribute, we've got the content, like you can see in the dev tools tab there. So get attribute content and you can see it's a list of those keywords really associated with the web page. So this is a string and we're just going to play around with it for a bit to see what we might be able to do with it and the type of functionality you can use once you get familiar with Selenium. So one of the things you might want to do is we've got this list of keywords. Maybe we can make it into a Python list, right? So let's do get attribute content and then we'll split it. So the default split splits a string on spaces. So this will give us a list of the keywords and a Python list of the keywords, not just a string list of the keywords. Okay, as always, remember to shut down your driver after you're done using it. But in this lecture, as you saw, we took a look at how you can get elements off of a web page using the name attribute in the HTML tags. Thanks everybody for joining and we'll see you in the next lecture. Hello everyone and welcome to our lecture on locating page elements in Selenium using HTML tag names. HTML tags are really the basic unit of web page setup and Selenium lets us locate page elements using their HTML tag names. So for example, A for anchor tags, H1 and H2 for header one and header two, form tags, button tags. We can really select any HTML tag in Selenium. Usually if we're finding page elements this way, it's more useful for finding all of the elements at once. For example, if we wanted to find all the links, we might search by the A tag for the anchor tag. If we wanted to find something more specific, typically a CSS selector or an ID or a name might be a more useful way of searching for those page elements. In terms of how you use this functionality, you have to start with all the imports and driver setup that we've seen before and then get your URL of interest. And then we can find the elements by the tag name. So we can write driver dot find element and then the first parameter will be by dot tag name. So we're searching by the HTML tag name and then we can pass the HTML tag of interest. Maybe it's an A for an anchor, div for a div, or a button for a button element. Additionally, you can use the same way. You can use find elements to find all of those HTML tag elements on the page if you're looking for everything and not just a specific one. Now that we've seen how to use this, let's head over to our Jupyter Notebooks to get some practice with using it in the real world. Alright, so here we are in our Jupyter Notebook and we've gone through all the imports. We've imported from our settings the Chrome driver path and the website we're going to take a look at is Books to Scrape. So we've imported the Books to Scrape URL and then we've set up the service, set up our web driver and now let's get the website we're going to take a look at. So we've got that Books to Scrape URL. So we'll get that, open it up on the page and start taking a look at things. So we're interested in locating elements by their HTML tag and what you'll notice as you look on this page is that there's lots of images because it's a book website, we're going to be buying books or something. So maybe we want to collect all the images on the page and the HTML tag for this and you'll see this if you use the dev tools to click on these things is that it's an image tag. So that's what's of interest to us. So we'll do driver dot find elements because we want everything, not just the first one by, we're going to be looking by tag name and the tag of interest is image. So let's hit that and now we've got all of those images and let's see maybe what some of the attributes on them are. One of them is alt like you can see here. So the alt submission, this is the alt text for the image. So for element in image elements and I'll only do the first ones or the first 10 or so and then we can print the element and we'll get the attribute and the attribute is that alt attribute. So let's do that and then you see we've got the alt text associated with each image and it's really just the title of the book. If you scroll through this page you'll see it's just the title of the book. Now the other thing maybe we want to collect is all the links and this might be interesting for us as we take a look at it. So we'll do driver dot find elements we'll search by tag name and we'll do it for A since A is the anchor tag for the links and then let's take a look at this list of link elements. So we'll do 0 to 10. Just take a look at the first ones and then let's print the... we'll do a link dot get attribute we'll get the title of the link it actually some links don't have titles so we can fall back on the text. So we'll do that for one line and then we can do the href let's get the href attribute as well. If you take a look at a link an href is really what the link is linking to, right? It's the URL that it would send you to. So let's print this off for the first couple of link elements on the page. So you can see here we can scroll up to the top so you can see the title here is book to scrape that's what we've got and it'll send you to the index home as well as the text so that'll send you to the index and so on and so forth, right? We get all these links here. Now let's do this for the last 10 links on the page and this way we'll get some of the... maybe some of the books as well since those have links. Okay. So now what you'll see here is something a little bit interesting, right? You've got kind of this weird list of things that isn't as nice as what we saw in the first couple of ones and if you look at the... let's look at this... let's look at this book here, for example. If you look at it, it's because really for each of these books we have two links associated with them. The image is wrapped in an A tag so we get that A tag and then let's click on this link here as well. We've got a different A tag there as well. So for each book we're kind of collecting two links which is why we get so many of those... so many of those different links associated with the same book. So again, that's something to be aware of if you're scraping on a website and you're pulling all the links, you might get double counts of things depending on what you're doing. And then as always, remember to quit your driver once you're done using it and that'll take you out of your automation. Okay, so in this lecture we took a look at how we can select items on a page using the tag name, the HTML tag name, in Selenium and some of the pros and cons of using that method. Okay, thanks everybody for joining and we'll see you in the next lecture. Hello everyone and welcome to our lecture on locating page elements in Selenium using the XPath. XPath is the language for locating nodes in an XML document. Because HTML can be an implementation of XML, we can use XPath to locate elements on a page. This is very useful for navigating a page when an element of interest doesn't have an ID or name attribute that would make it easy for us to select in Selenium. We can use XPath to search in relative terms, which is really the more advisable way of using XPath, or we can use it to search the web page in absolute terms, which is not really as advised. Here's a few examples of the syntax of XPath in case you haven't seen it before. So to describe the absolute path to the first input on a page, we could write slash HTML, slash body, slash input, and then one within brackets. That specifies the first input on a page and the absolute path to that input. If we just wanted to use the relative path to find the first input, we could use double slash input and then one within brackets. That'll get us that first input on a page. If we were looking for a specific input with an ID, we can use the at sign to specify that HTML attribute. So here we've got double slash input and then within brackets at ID, so that's specifying that HTML attribute is equal to search box. So that'll help us find an input with an ID of search box. And finally, if you want to do that relative search around the page that XPath can sometimes be useful for, you could write double slash A, then within brackets at title, so that's the HTML attribute of title, and we want the link where that is equal to the string title. Then we can go up by writing a double slash, or a single slash, sorry, double period, and then another slash and div, and so that'll give us the div that's one parent above the link with the title of title. If some of those examples were unclear to you, just be aware that you don't have to be fluent in XPath for this course. That's not really what we expect. What's more important is that you're able to understand what element or what elements an XPath is locating on the page and how XPath works in practice. The Chrome DevTools lets us copy the XPath from the sources tab, which makes our life a lot easier anyway. And you can also test any given XPath in the console tab of Chrome DevTools. So with all that said, even if you don't have experience with XPath, we'll be able to figure it out as we see some examples. In terms of how you locate a page element practically in Selenium using the XPath, first you have to go through all the imports and setup that we've seen before, then you get your URL of interest. And to find the elements with the XPath, you can just use driver.findElement. The first parameter will be buy.expath, and then you can pass the XPath string. Again, you'll usually not be writing this yourself, you can just copy it from the Chrome DevTools. If you want to find all the elements, again, you can just say driver.findElements, first parameter is buy.expath, and then you pass that XPath string. Let's head over to our Jupyter Notebook to get some practice with this. All right, so here we are in our Jupyter Notebook. We've got all of our imports that we need, so we've got the WebDriver, the service class, the buy class, and then from the settings file, we import the Chrome driver path as usual for the service, and then we're going to take a look at the Books to Scrape URL for this lecture. Start up the service, start up the driver, and so that's what we've got over here with our Chrome window open, and then we can get that URL that we're going to take a look at. So driver.get, Books to Scrape URL, run that, and it takes us to the page. So to start off with, we'll start off with a pretty simple XPath, and as a good tip to remember, if you want to test your XPath strings before you use them in Python, you can test them out in the console here in Chrome DevTools. So if I wanted to get all the links on this page, I could do $x, which is how you check for XPath, or run XPath strings in the console, and I can do slash slash a, and that'll get me all of the anchor tags on the page. So that's an example of an XPath, and maybe I want to get all the all the anchor tags that have an attribute of, an HTML attribute of title. So then I can inside brackets after the a, I can write at title, and that'll show up as there's 20 anchor tags on the page that have that attribute. If we get on here and we click on the title of one of these books, it's really just the title of the book on the page. So you've got the picture, and then you've got the name of the book, that link is what has the title attribute. So we can try this out in Python with Selenium. So we can construct something called, let's call it list elements, to find all those book links. We can do driver.find elements, because we want all of them. We're going to be searching by XPath, and then we can just use that XPath string that we just wrote, so slash slash a at title within brackets. And that'll give us those 20 link elements. So if I look at the length of that, there'll be 20, which is what we should expect. And then you can get that title off it as well. So link elements, if we want to get the first one, so zero, we can get that HTML attribute of title off it, and it should show us what we see here, so a light in the attic. That's the name of the book. And as we mentioned in the lecture, where XPath really shines compared to some of the other methods of finding things in Selenium, is when you're locating things that don't have a good identifier, but you can find them relative to other elements. So one example of this is the price. So the price here is in this P tag with a class of price color, but you probably want to associate it with the book too. And just looking at it in the HTML, there's not really a good way to do that. So what we might want to start with instead to back into getting that price is to start with the title of something that we're looking for. So here we've got the a tag with the title, so we can start looking for that. So we can start off, let's do this in the Chrome DevTools. So we can start off with slash slash a, so we're looking for that a tag where the title is equal to... What's the title? A light in the attic. And make sure to close that with a bracket. Let's see. Dollar sign x, not x dollar sign. Dollar sign x, and that gives us that one anchor tag object. Okay, so now if we go back and look at the elements, where is that in relation to the paragraph that we want to pull out here? So the a tag is within this h3, so we have to go up one, and we still can't get to the p tag from this h3, because that's self-contained. So we have to go up another level. So we actually have to go up to this article class product pod. And the way we can do that with xpath is we can have some double periods. So that's up one, so that gives us that h3, up two, that gives us the article with the class of product pod. From there, we can get the div with the product price. So here we've got this div that contains the p class of price color, and it's got a class of product price. So that's the branch we want to go into, so to speak. So let's reuse that. Then we want to go into the div with a class of this is equal to product price. So run that. So we've got that div of product price. And then from there, we can just get the first paragraph tag. So let's run that string again. And then we've got one last thing we're looking for. So we can just do p. And we want a p where it's got a class of the price color. Let's just check that on the elements tab. Yeah, so the price of the book is here in this p tag with a class of price color. So we run that on the console. You see we get this p object here. That has a whole bunch of stuff associated with it. But what's important for us is that we can use it. We can now use this string in Selenium, in Python, to pull this element. So if we want to find the price element, we can say driver.findElement. And then we're going to be searching by an x path again. And the string of interest tab this over, because it's a long string. We can just pull from the console. So let's tab this down, copy this. So we don't have to type it again. And stick it in there. And so you can see it's quite a long string. But this should work to find it, because we tested it in our Chrome dev tools. So let's see, end of line. OK, so we need to remember to end the string, and end the function, or end the method. Let's see. OK, so what it was, was all there was, the parentheses was in the wrong place. So now we've got this x path string. So let's run this. And so now we've got that price element. And we can check out the text within that, which should give us the price of the book. So we do price element dot text attribute. It'll show us that 5177 pounds for the book. And now if we wanted to, we could strip off the pounds, use the numbers to run some sort of analysis, or whatever we want to do. OK, so in this lecture, we went through how you find elements by x path in Selenium. And we also saw the useful ways that you can use the Chrome dev tools console to check your x path before you use it in Python. Thanks for joining, and we'll see you in the next lecture. Hello, everyone. And welcome to our lecture on locating page elements in Selenium by their link text. Anchor tags on a web page will usually have some user-visible text showing on that web page. Searching using this information is useful when you know some or all of the link text within the anchor or a tag. Selenium lets us search with the exact link text. So for that, we can use by dot link text, or we can also search by a partial string match. So for that, we can use by dot partial link text. To use this functionality, we'll have all the imports and the setup for our driver that we've seen before, get the URL of interest, and then find those elements on the page. So we can use driver dot find element, and then pass by dot link text as the first parameter, and pass the link text of interest. On the other hand, if you maybe want to find or search by a partial link text, so a partial string match, you can use find elements, pass by partial link text as the first parameter, and then pass the link text of interest as the second parameter. Now that we've talked about how this works, let's head over to our Jupyter Notebook to get some practice. Okay, so we're in our Jupyter Notebooks, and we've got our Chrome window open already. We've done all of our imports, imported the Chrome driver path, and the website we'll be using for this lecture is Books to Scrape URL, so we'll be using that. So let's pop that open, use driver dot get, and then Books to Scrape URL, run that, and that'll open that page up. And so in this practice, we're going to be locating page elements by their link text. So maybe one of the ones that we want to locate is one of these books. So we've got tipping the velvet here. This is a link. This is an A tag, and it's got text inside it. So we can use that in Selenium. So we can find that link based on just the link text. So link element, we can call it link element. We'll do driver dot find element, and we're going to be finding it by link text. And the link text is just what you see there on the page. So it's tipping the velvet. So let's run that, and okay, we've got the link selected, and now we can check things, and the link text is really just what we were looking for, right? It's just tipping the velvet. It's just the title of the book. The tag name is just going to be an A tag. It's just an anchor tag. So that's all well and good. That's what we expect it to be. As mentioned in the lecture, you can also find links based on partial text. So let's say you want to see how many titles or how many books on the page start with SH, for example. So we can say, I'll just call this add elements, driver dot find elements. So we want all of them, since we're searching by partial link text, partial link text, and then here we can put that SH. We just want to look for the titles that begin with SH. So let's run that, let's enter, we got those elements. And then if we just do, let's make this a link, or make this a list, rather. Let's do lm dot text. We can do a list comprehension. lm dot text for lm in that add elements object. And so here you see we've got short stories, sharp objects, and Shakespeare sonnets are the three books that start with SH. Okay, so in this lecture we saw how you, how you find elements on a page using link text matching or partial link text matching in Selenium. Thanks for joining, and we'll see you in the next lecture. Hello everyone, and welcome to our lecture on chaining locators in Selenium. In the past few lectures, we've taken a look at a bunch of different ways that you can select page elements in Selenium. Things like using a CSS selector, using an ID, using a name. But all the locators we've reviewed can be chained together as well. This is sometimes useful for when you're navigating between parent and child elements that may be hard to access otherwise, or hard to specify because maybe they don't have a name or an ID attribute. It also might be more readable to select page elements in this way rather than using an X path in certain cases. As an example of how you might do this, first you need all your Selenium imports and the driver setup, and then to get the URL of interest to you. And to find elements with chained locators, you could, for example, select an element one, so using driver.findElement, and then select that element by a CSS selector, passing a CSS selector string. So now you have that element one web element. Then in the next line here, you could use findElements on element one, so on that page element that you selected with a CSS selector. You could use findElements on it and search for something by a tag name, for example, to find all the A tags within that portion of the HTML page. In this way, you can break down an HTML page and maybe make finding elements within the page a little bit easier. Let's head over to our Jupyter Notebook and get some practice using this functionality. All right, so we're in our Jupyter Notebook. We have all of our imports from our setting file. We've got the Chrome driver path and the website we're going to take a look at is the books to scrape URL. So it's already open here in the Chrome window. So you can just do driver.get to get that URL. And then we can get going with taking a look at how to chain these web element locators to find specific page elements. So like we saw in one of our previous lectures, sometimes certain locators, such as ID or name, are not really available for specific elements. And we might have to use XPath instead. And the issue with XPath, as you saw in a previous lecture, is that sometimes the syntax can be complicated, the strings can be quite long. So another option for us is that we can chain these locators together and maybe make our code slightly more readable. So the first thing we're going to take a look at is let's take a look at these books. So if we look at these books, you'll see that really all the information about the book is contained in this article tag. So that's where the name is, the picture, the price, all of that. And so you have these list items, and in each list item there's this article that's really the container for the information. So this is a good place to look if we want to do what we were doing in a previous lecture where we were looking for the price of a book. So first things first, let's get all the book elements on the page. So we'll do driver.findElements. And we'll do this by CSS selector. And our CSS selector in this case is going to be that article. So that's the tag and then the class is product pod. So we get that and we'll get a list of list of all the book elements. So like we saw before, there's 20 book elements on this page that we can start to take a look at. Now since every element in book elements is a web element, we can use findElement or findElements on it just like we can use with drivers. So we're used to doing driver.findElement or whatever. But you can also use, you know, for each item in book element, you could do bookElement0 for example, and then do a findElement there as well. So bookElements0, so that's the first one. You can find an element and let's do by tag name. Let's get the image maybe. We can get the image out of that section of the HTML. And maybe we want to get the, let's get this on a new line just to make it easier to read. We can get attribute and let's get that alt text. So you can see here we're finding the tag name, we're finding the image. And the image is here within that a tag and it's got the alt text that's the title of the book. So if we run this, we'll see a light in the attic. So that's the first book and that's the image and the alt text off that image. Okay, so that's one example of how you can change, chain those findElement methods together. Another one we might do, like we were doing in a previous lecture, we're taking a look at getting the price, right? So let's do that for this first book element. So we'll find element and we'll do it by CSS selector. And what we're looking for here is a paragraph tag. So let's click on this to see where it is. So it's a P and it has a P of price color. So it's a paragraph tag with a class of price color. So P dot price color should give us that information. And then if we get the text because it's inside the P tag. So you find that, okay, all you know, we've got the price that we're looking for. And in some sense, this is maybe easier to read than the X path we were using before. Even though the X path is maybe more direct, you might say, or kind of simpler because you don't have to change things together. But now what we can maybe do is scrape this page to get all that price information associated with the book. Because as we saw before, there's nothing on this price that directly associates it with the book. You have to go up the HTML tree. So I'm going to expand this for a minute so that we can see the code because the code is a bit long for this part. So we'll instantiate a price list here. And so we've got our list of book elements. So for book in book elements, what we can do is we can do price list dot append. So we're adding to that list. And we're going to add, we're just going to make a dictionary of this price and title information. So we've got a title and then we've got a price. And we can just use what we were using above to get this information. So the title comes from that book element. On the element, you can find another element by the tag name. We're going to find the image. And then we're going to get the attribute of the alt text off the image. And that's going to give us the title of the book. In terms of price, like we saw above, you've got the book element. You can find an element within that section of the HTML. We're going to do that by CSS selector. And it's P dot price color. So P tag with the class of price color. And then we want the text because it's what's inside there. Okay. And so this should be able to do it. Okay. So the for loop is done. Now let's take a look at our price list. And you see here we've scraped all the information that we maybe wanted to get. So a light in the attic, we've associated it with the price. And if we go back to that web page, it's the price we see on the web page, right? So this is an example of how you can chain these locators together, chain these find element methods together to find information on a page. So let's just walk through. Let's put this over to the side again. Let's just walk through what we're doing here. So we had our list of book elements, which was really all of these article classes, right? Or article tags on the page. Then with the each of these article tags to get the, to get the book title, we went to the image, found the image there, took the alt text, and then to find the price, we went down to the p tag with a class of price color and pulled the text to get the price. And we weren't worried about getting multiple ones because there's only one in each of these article trees or each of these article containers. So again, that's it for this lecture. We learned how to chain locators together to find objects in a page. Again, a good alternative if you don't want to use XPath to find things. Thanks for joining, and we'll see you in the next lecture. Hello everyone, and welcome to our lecture where we'll talk about some of the useful web element attributes and methods in Selenium. When you find an element on a page, on a web page, and you access it in Python using Selenium, what you're doing are accessing web element objects in the Selenium package. Because of this, it's a little bit useful to know some of the more common attributes and methods associated with these web element objects. If you'd like a complete list, I'd recommend you see the documentation because that's where you'll find more information and the most up-to-date information as well. In terms of some useful web element attributes, with any web element, you can access the text. So the text inside that web element on the page. The tag name, so this will return the HTML tag associated with the web element. The size, which will be a tuple for the pixel size of the web element on the web page. And then you have these three, what are really methods, but I think you can consider them attributes at the bottom. So is displayed, is enabled, or is selected. This will all return true or false values whether a web element is displayed on the page, whether it's enabled, or whether it's selected. In terms of some useful web element methods that you might use in your Selenium work, you can screenshot a page element. You can use find element or find elements, as we saw before. So if you have a web element selected, you can change those find element methods together to get deeper into the HTML tree. Then if you want to get certain attributes or properties from that page element, you can use get DOM attribute, get attribute, or get property. We'll talk about the difference between these three in a minute. And finally, you can also get the value of CSS properties using value of CSS property. So if you wanted to know the border or padding or the color of an item on a web page, you could use that method on your web element. In terms of the difference between get DOM attribute, get attribute, and get property methods, I like to think of the get DOM attribute as getting an attribute from the underlying HTML since that might change with JavaScript or jQuery, for example. Get property, on the other hand, gets the current state of an attribute that might change. So let's say you have text in a text box, for example. Finally, get attribute gets the initial content element of an attribute that might change on a page. So instead of getting the text in the text box, you could get the placeholder or the initial value of the text in the text box. Now that we've talked about these methods, let's head over to Jupyter Notebooks to get some practice with using them. All right, so we've got our Jupyter Notebook open, we've got our service instantiated and our driver set up, and we're going to use the books-to-scrape URL for this practice. So you can use driver.get, books-to-scrape URL, and I'll take you to this page. Now, to review some of the attributes and the methods that are on the web element objects, we can just get something to work with, kind of arbitrary. So let's do this first article tag with a class of product pod, so that has all the book information for this first book. So we can just say, call this book element, driver.findElement, we're going to be searching by CSSSelector, and then what we want is article.... What is it? ProductPod. Article.productPod. So that's the class. So now we've got that first book element. So one of the things we might be interested in is maybe the text. So what's inside this web element? So if you look at it, it's got a couple of things here and it's all this stuff that you see here, right? A light in the, the pound sign and the price, whether it's in stock, add to basket. So that's all the text inside that book element. So that's one attribute that might be of interest. You can check the tag name like we saw before. So this will give you the name of the HTML tag that the element is, or the HTML tag associated with the element. You can check out the size. So let's do bookElement.size. And that will give you this dictionary of height and width in pixels. What else can we take a look at? Then you've got some of these Boolean, they're really Boolean methods, but I think kind of think of them as, as attributes as well, because they tell you something about the, the web element that you have. So bookElement is displayed. So it is displayed on the page. BookElement you can do is enabled. That's true as well. And what else is, is selected as the other one. So you can do that. And that's false. So it's not selected. It's not like a form field or something where you'd be selecting it. So this is a, this is just a quick overview of some of the attributes that might be useful to you when you're working with, when you're working with web elements. So now let's take a look at some of the methods that we can use with these web elements. So we've got our book elements. One of the things that we can do is we can do screenshots of objects or of web pages. So we do screenshot this page. And all you have to do is pass a parameter for the, for the file name. So I'll just call this test.png. And so that should be downloaded. We can take a look at it if we want. I won't do that now. I'll take a look at it later. What else can we do? You can get, we can chain locators. So let's take a look at that button element. So we've got the, and the button element that we're going to take a look at is this add to basket there. So let's do book element.find element. And then we're looking by CSS selector. And we want to get the button out of there. So the button element.text should be add to basket like we see there. That's an attribute again. Let's do some of these methods, some of these methods. So button element. Let's get the DOM attribute first. So type. And let's click on this just so that we can compare it to the HTML. So here the type is submit. So let's get that. So it says submit. Button element. Let's check out the get attribute. Get the type attribute again. It says submit. And then we can get the property. Let's try get property. The get property method. And get name. So if you notice there is no name associated with this. So it should return just an empty string. So we've got submit. We've got class. We've got data loading text. But those are the only attributes that we have there. So which one of these that you want to use or the one of these that you want to use will depend on what you're looking for and the stability of your selenium code really. So if you find one that works well that's the one to use. And then the last method to take a look at is the, let's take a look at value of CSS property. So if we want to know the color of the button. This will give us that string which is an RGBA of what the color of the button is. Or we can do maybe let's value of CSS property. Let's look at the padding here. Again, that's just another CSS property you could take a look at if it interests you. If that's something that you want to take a look at for whatever reason. Okay, so we went through some of the methods and some of the attributes that are associated with these web elements. Things like tag names. Things like getting the CSS values or getting properties or attributes. This is a brief overview if you want the full list of attributes or methods associated with these web elements. Take a look at the documentation. But this should give you a good overview of some of the useful ones and some of the ones that we've used in previous lectures as well. Okay, thanks everybody for joining this lecture where we talked about the useful methods and attributes on the web element objects. And we'll see you in the next lecture. Hello everyone and welcome to our first guided project for this Selenium course. Our first guided project will involve building a web scraper for Wikipedia. Since we've gone through the section on how to locate page elements in Selenium using things like CSS selectors, IDs or names. Our goal for this project will be to use that knowledge to scrape link information from four of the main div sections on Wikipedia. The four sections of interest to us are the from today's featured article section. So that's a Wikipedia article in short form on the main page. There's a section called in the news which has bullet points on news items occurring around the world. Did you know which is a section with facts and figures from various things and various Wikipedia articles? And then on this day which is a record of historical events that happened on the day of interest and their articles, the links to their articles on Wikipedia. If you need the link, the link is shown on the second bullet point here, but you'll also be able to pull it from our settings file or from the GitHub repo. In terms of what the main page looks like as we go through the project as well, these are the four divs of interest, or the four sections of interest to us. So like I mentioned before, we've got the from today's featured article in the news did you know, and on this day we're going to use what we've learned about the element locators to construct a dictionary of links by the div section. And we want to store the following information on each of these links. We want to store the title of the link, the text of the link, which can be different from the title, and then the href, which is the link to the Wikipedia page with more information. Our dictionary structure will look something like a string with the div identifier, and then we'll have a list, and in the list each item will be a collection of those three pieces of information. So a title, and then a string with the title, the text, and then a string with the text, the href, and a string with the href. You might be asking why we're going to go about scraping data this way as opposed to using something like beautiful soup. And first off, it's really good Selenium practice. You'll have to locate web elements regardless of what you're using Selenium for, so we might as well get some practice using those skills. And second, and this is something that you may have noticed on some websites, is that you may want to scrape data that can only be accessed via Selenium. You can't get it via an API or via just a basic HTTP call. For example, you might need to fill out a form first in order to get that information. So this is good practice for using those skills. Okay, now that we've talked about what we're going to be doing, let's head over to Jupiter and start coding. Okay, so we've got our Jupiter notebook open, we've got all the imports that we used in the previous section, so the web driver, the service, the by class, from our settings file we'll import the Chrome driver path for our service, and then we're going to be scraping Wikipedia, so we've got the Wikipedia URL as well. We start up the service, start up the driver, so we've got the Chrome window open here on the other side, and now we can get that Wikipedia URL, so driver.get, Wikipedia URL Shift-Enter to open that up. Now let's head over to this page so we can open up our Chrome tools and dock this to the bottom. Control plus to make the code a little bit larger. Easier to read. And now let's start taking a look at this page to see how we can scrape the information from these four sections. So we've got today's featured article in the news what else. Did you know and on this day, so those are the four sections we want to scrape through. Now if we click on this, we can click on the span just try to find out where exactly we are. So we've got this div that's mp-left, but if you scroll down actually I think this contains everything. So we need to go a little bit more specific than just this mp-left which is main page left. And actually what you can see here is we've got this div under the main page left that's got an ID of mptfa. So that's mptfa that's today's featured article. So that's the div of interest to us that we want to select there. And if you keep going on the page you'll see some other ones. So mp dyk if we click on that, that's down here below. So did you know? That's mp-dyk so main page did you know? And then we need to also go over here so we've got div with an ID of mprite again that's going to contain both sections so we need to go more specific. If you look in here you've got a div mp itn so that's in the news and then div id mpottd so main page what is this? Main page on this day. So those are the kind of the four IDs that we want or the four CSS selectors that we want. So let's go back to our Jupyter notebook and start writing these constants out. So we're going to use CSS locators you could use IDs that's fine too I'm just going to use CSS locators for this so we've got a div which is mp tfa from today's article tfa what is that? Yeah tfa okay tfa today's featured article so let's call this tfa instead so that matches up then we have itn so itn CSS locator I'm just naming these ways because they're constants so that's more obvious div hashtag mp itn then we've got the dyk so did you know CSS locator that's div hashtag mp dyk and thenottd so on this day CSS locator is equal to div hashtag mpottd and then since we're going to be running through these in a for loop we can just add them all to a list so let's call it locator list and let's do tfa itn what's dyk andottd so those are the four sections so run that we've got all of our constants that we need we're going to be building a dictionary with information so let's do link dictionary just instantiate that as a python dictionary object and when we loop through each div what we want to do is we want to initialize an empty list for storing the links find all of those link tag elements and then pull out the title text and the href and store that and add that to the list of dictionaries with that link information to our larger dictionary with the information on each section so for each locator in our locator list it's what we're going to be looping through first first we have to find the right div so we can say the div element of interest is going to be driver.findElement because we just want the one div and we're searching by ID or we're searching by CSS selector with an ID so there's only going to be one anyway and then we've got a locator so the locator is that string constant here right div mptfa that's going to find that div for us so we've got our div selected and then for each of these we want to up a list so we'll start with an empty list here ok so within the div we want to look for those anchor tags right so for link in each div element what we want to do is we want to find elements plural because we want all of the all of the a tags so by.tagName we want that a so that's the start of our for loop and let's make this big for one second while we're coding this out ok and then so for each link we can start to pull the info from it so the info dictionary that we want is going to be a few different things like we talked about so we want to get the title we want to get the text and we want the href as well from it so for the title we can do link so each link is going to be a web element which means we can use get attribute and the attribute we want is that title attribute of the link so get attribute and then the parameter is a title string for each link we can also get the text so that's just the text attribute we can pull off and then the href we can also use get attribute since it's a web element it'll have that href we can just get attribute href and then once we've got that dictionary constructed we can add it to our list, our empty list there so it's going to be a list of dictionaries info list.append info dict so that's what's going on in that inner for loop finally the last thing that we can do is add this list once it's fully constructed to that link dictionary so link dictionary then we can add the key right so the key is going to be the locator that locator string and then we can just add the list that we've built up okay so let's walk through what this is doing again so for each locator in the locator list the locator list is just the list of that those strings with the css locators for each locator in that list we're going to find that element so that that part of the DOM really then we're going to instantiate the empty list to store the link information and for each link that we find within that div we're going to build the dictionary with the info with the information so title, text and the href we're going to add that dictionary to the list of dictionaries that we're constructing and once we're done looping through all the anchor tags we'll add to this link dictionary in the locator key spot so for each one of these divs a list of those links and their information so let's run this now might take a minute there it goes still running okay so now the link dictionary construction is done so let's take a look at the type of data that we've uh that we've collected here so we're going to do let's just look at the today's featured article css locator so if I run that you can see we've got all this stuff title text sometimes it has text sometimes it doesn't and then the href of interest so battle of the defile umayad khalifate uh let's pull this over so that we can take here and you can see on the screen here we've got everything that we're kind of that we were looking to find so we've got battle of the defile that's this link here umayad khalifate that's that link here so on and so forth so we've got all that so let's scroll down a little bit go to this new page and maybe let's just see how many how many links are in each section so we'll loop through that dictionary so for key value in the link dictionary uh and you can do this with items so dot items method on the dictionary let's just print and we can use a nice f string here for formatting let's just print key and do do a function here so length of the value uh and then make it more obvious links so each key in the link dictionary is going to be that string uh with the div id in it and then the value in the dictionary is the list of dictionaries with link information so if we just do a length on that list we'll get the number of links uh in each section of the page so let's run this okay so you can see here we've got in the the featured article there's 16 links uh itn is in the news so there's 21 links dyk did you know there's 16 links and then on today what is it called on today on this day in history uh there's 38 links on that section okay so just to go through and review what we did in this guided project since we're at the end of the guided project we really ran through a way of using everything that we've learned so far about locating page elements in Selenium so we started up our service started up our driver got the URL of interest to us uh we specified all these CSS locators after we went through the DOM or the HTML here to figure out where things are and which IDs we should use compiled it into a list and then we ran through that list to pull out link information or information on each links using some of the methods and attributes that we talked about in the previous section and once we had done that once the link dictionary had been constructed then we just kind of took a look at it uh to see to get some more information about the web page that we just scraped as always remember to quit your driver once you're done with it so you don't leave it hanging uh but that's about it for this guided project nice job everyone on working through this thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on opening and closing windows in selenium in older versions of selenium we had to instantiate a new web driver object in order to open a new window but now in the newer versions of selenium we can switch to new windows much more easily so the way we do this is with the method driver dot switch to dot new window and then we specify the string of window this is important as we'll see in the next lecture because you can also use this to switch to a new tab if you'd like to close your secondary or any active windows you can use driver dot close instead of driver dot quit just a note on this method though it doesn't work if there's only one window active so if you try to use it you'll throw an error in terms of what this code looks like if you're looking at it among other bits of python code you can do all your imports and your service and driver setup then you can get a url of interest so here it's just first url.com to switch to a new window you use the method we just talked about so driver dot switch to dot new window window then you can get that second url and it'll open up that second url in the new window to close that new window you can use driver dot close like we mentioned before and then it'll take you back your active window will become that window with the first url in it in terms of the old method if you see this you might see this in older selenium code bases to open up the new window you had to instantiate to driver so you'd have driver one which is just your web driver and driver two which is just another web driver and then you could use that get method to get the first and second urls of interest and if you wanted to close those windows you had to use driver dot quit as we normally do okay now that we've talked about opening and closing windows in selenium let's go over to our jupiter notebook to see how it works in practice okay so here we are in our jupiter notebook and we've done our imports that we need and then from our setting file the websites we're going to take a look at we're going to take a look at the books to scrape url and then the python url as well so those are the two urls you're going to want to import from the settings file we'll set up our service set up our driver and then we can get started so the first things first let's get our website so first we'll go to books to scrape go to that url so that pops open in our chrome window now so if you just use driver dot get and let's get the python url you're just going to go to that url you're not going to open up a new window so like we saw in the lecture what you're going to want to do is use driver dot switch to and then new window and then you can specify a window here as the parameter so this will open up a new window like you can see here now we've got this new chrome window in front of the old one and so then I can get a url if I use driver dot get books to scrape url then I'm going to get the books to scrape url in this new chrome window not in the original one if you want to close the active window without quitting the web driver you can use driver dot close and this will close that books to scrape website and window so driver dot close now we're back to the python way of doing things or the python website rather and just in terms of you being aware of it as well like we mentioned in the lecture an older way of doing this before we had switched to new window might be to just instantiate a new web driver so now we can use driver dot chrome again web driver dot chrome the service is still the service we just named it driver 2 you can get a new web page here as well as you can see and then you can get that maybe get that python url driver 2 dot get python url and that'll get of us that that python website in a new window so that's two ways of doing it switch to new window is probably the best one to use in newer versions of selenium but the old way is just so that you're aware of it then in terms of closing this we can quit our second window so driver 2 dot quit we'll close that and then we're good to go okay thanks for joining this lecture where we saw how to open and close new windows in selenium and we'll see you in the next one hello everyone and welcome to our lecture on opening and closing tabs in selenium opening and closing tabs in your browser in selenium works pretty much like what you do to open and close windows like we saw in the previous lecture so to open a new tab what you're going to want to do is use a method that's driver dot switch 2 dot new window only instead of writing window as the parameter you can write tab in order to close any tab of interest you can use driver dot close but as we mentioned in the previous lecture this only works when there's more than one tab or more than one window if you only have one active window or tab that means you're going to run into an error if you try to use driver dot close in order to switch between tabs or to switch between windows as well we can use something called window handles these are unique IDs for windows or tabs in your current selenium session to get the current window handle or ID you can use driver dot current window handle if you use driver dot window handles this gives you a python list of all the window handles from your current session if you want to switch to one of them which is basically switching between windows or switching between tabs you can use driver dot switch 2 dot window and then pass the window handle as the parameter and usually the best place to get this is from that list of window handles the old method for opening a new tab is something that you might see if you're working with an older selenium code base but basically what you had to do to open a new tab was to execute javascript via the dot execute script method in selenium so in order to open that tab you'd write dot execute script and then within there you'd write this javascript which is window dot open with a url parameter and that will open a new tab with the url that you pass the driver dot execute script method is a generic method for executing javascript and we'll see a little bit more of it later on in the course but for now it's not terribly important to worry about as mentioned we don't expect you to be a javascript expert for this course just to be able to read it and understand what's going on in terms of what this looks like so by clicking the code you'll have all your imports and service and driver setup then you can use driver dot get to get your url and you can switch use that driver dot switch to dot new window tab to open a new tab once you have that new tab active you can get whatever url you're interested in and you can use driver dot close to close that tab finally as I mentioned to switch between tabs what you really want to do is you can use driver dot switch to dot window handles one so that gets the second item in that handles list to switch to a different tab or a different window depending on whether you have windows or tabs open then you can switch back using another window handle so here it's handle zero which is the first window handle or the first window id and to check which window you're on you can print the driver dot current window handle which will give you that id of the current window as we'll see in the exercise for this lecture these window handles tend not to be very user readable or user friendly so you should go off what your screen looks like to kind of figure out which which tab you're currently on and then if you want to close the tab you can use driver dot close which is what we saw with the windows as well now that we've talked about how to open and close and switch between tabs in selenium let's head over to jupiter to get some practice alright so here we are in our jupiter notebook we've got our imports from our settings file we're importing the chrome driver path the two URLs we're going to take a look at our books to scrape and python set up our service set up our driver so we've got the chrome window open there and now let's get our first URL so we can just do driver dot get then we'll get that books to scrape URL so that will open up in the page now if we want to open up a new tab we can use driver dot switch to new window and then just specify tab as the parameter and so we've opened up that new tab and then we can do driver dot get python URL we can get that new website there in our browser now the old method of doing this if you wanted to open a new tab would have been to execute some javascript so execute script here and then you could do you could have done window dot open just like that and that should open up a new tab there so like you see we've got a new tab but that's an old way of doing things it's much better to use switch to new window and tab then to run javascript or to have to run javascript if you want to switch between windows what you're going to have to do or switch between tabs to tabs or windows what you're going to have to do is check out the current window handles so this is how you switch between things so current window handle it will give you the ID of our current window or our current tab so that's going to be the ID for this python website if we want to look at the URL we can check that look at the current URL it says python dot org so that's good and then you can list all the window handles which are all of those IDs for each tab or each window using driver dot window handle so here you see we've got two of these things now if we scroll up and look at the old one so A96A A96A is that second tab or the second value here so the other one DE9E would be the first tab there which is the script website so if we wanted to work with this for a little bit let's just do I'm going to do import scroll this up a little bit I'm going to import time here so that we can use time dot sleep for a second so for handle in driver dot window handles let's just switch between them driver dot switch to new window or no switch to window and then we can specify the window it's just going to be that handle string and then we'll print let's see make this an F string now on driver dot current URL so that we know which one we're on and we'll just do time dot sleep make this three seconds okay so now we're on books to scrape and now it switches back to python.org so that's what happens we just cycle through some of those window handles in order to take a look at our various tabs now if you want to close a secondary tab you can use driver dot close just like we did with secondary windows so this will close our current tab but then if we try driver dot close again this is going to throw an error at us because you can't use it with a with a non or you can't use it with your primary window if there's only one window one tab open so if you want to do that now you have to use driver dot quit as always okay so in this lecture we went through how you open up new tabs in python using switch to new window and then how you can cycle through your tabs using window handles and then using switch to window and using those specific window handle strings or ids to switch between the tabs okay thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on navigating iframes iframes or inline frames are basically html elements that load other html elements inside of your web page one way of thinking of it is it's kind of like having html within html the elements inside an iframe in terms of selenium can't be selected directly when you open a selenium web session on a web page instead what you have to do is you have to switch from the main or outer html document into the iframe of interest additionally we have that same problem but in reverse once we're inside of the iframe we can't access the main html or the outer html from our selenium session we'd have to switch back to the outer html in terms of the steps of navigating an iframe or navigating iframe elements in selenium the first thing you have to do is find that iframe within the html document or within the DOM then you can switch to the iframe using driver.switchto.frame and for this you can pass a web element that specifies the iframe of interest then you can find your page element of interest within the iframe so maybe that's a div or an anchor or a form element whatever you're interested in and then to get back to the outer html you can use driver.switchto.defaultcontent and that'll take you back up to the highest level of the DOM or the highest level of the html page if you will in terms of what the code looks like once you get your URL of interest if you want to find something that's inside the iframe so let's say you'd use driver.findElement and you have an id id string for that element inside of the iframe if you try to run that code it'll actually fail and you'll get an error so first you have to find that iframe element and we can just use driver.findElement to select it then we'll use the method we talked about so driver.switchto.frame and then we can pass that iframe element in order to switch into the iframe once we're inside the iframe we can look for that element we were looking for by id so driver.findElement by id inside element and we'll actually select it successfully unlike when we were outside the iframe if we want to get back to that upper level of the html or the DOM we can use driver.switchto.defaultContent if this seems a little bit confusing it'll look a lot more clear once we have a chance to go through our jupyter notebook so let's head over there now to get some practice alright so we're going to be taking a look at how we navigate with iframes we've got our imports all set from our settings file we're importing the chrome driver path and the url we're going to use here is the jQuery url or the jQuery ui url rather so you can see up here it's jQueryui.com or you can go to the settings.py file to take a look at what the url actually is set up the service set up the driver and then the url we're going to get actually is the jQuery url plus draggable so it's this slash draggable page on the jQuery ui website and the reason we're going here again you don't have to know jQuery for this for this course it's just that there's a good iframe good example of an iframe here so now what I can do let's do a find here let's see if we can find an iframe and here we can find an iframe on this web page so if you look at this we've got this iframe here it's a class of demo frame and then within the document there's this html here so there's different html from what's outside it's got a head it's got a body let's take a look at the body and we've got this div and it's got an id of draggable to it so what we can do is we can try to try to select that that div we want to get something out of that that div within the iframe just so that we can see that you're going to get an error when you try to do that so let's do driver.findElement we're going to be searching by id and the id of the element is draggable so we can just do draggable here and if I run that we should get an error that throws up it's going to show something down here no such element unable to locate element id draggable and you probably think to yourself well we've got this div it's got an id of draggable why can't we select it so we can't select it because first we have to switch to the iframe so let's get the iframe element first so iframeElement let's find it it's going to be driver.get no driver.findElement by let's do cssSelector because I don't think this has an id I think it's just a let's take a look here yeah just as a class demo frame so we can do find by cssSelector so this iframe with a class of demo frame so now we've got that iframe element there okay so we've got the iframe element and we need to switch to inside the frame if we want to get that that div that we're looking at with the with the id that has the draggable id so we're going to use driver.switchTo then we're going to do frame and we're going to do that iframe element so now we're inside the iframe there so now if I want to do that again so let's try driver.findElement by id the id is draggable let's store this as a variable draggable element if I run that then I can get it right now I don't have an error I've got it in there let's see a tag name it's a div right okay we've got everything that we like there but now what we might want to take a look at let's look at this navbar so we've got a nav id main this is outside the iframe let's try to select that and what you're going to see is we'll run into an error so we can do driver. let's see driver.findElement by id we'll just find that main that nav of the main id so let's try to store this as a variable so what is this main nav just call it that driver.findElement and you see we get another one of these error saying no such element unable to locate element again that's because we're inside the iframe so we want to move outside the iframe so now we can do driver.switchTo and we're just going to switch to the default content there now we're outside the iframe so if we want to do main nav it's going to be driver.findElement by id the id is main we run that now we've got it main nav that tag name it's a nav tag and now we've got that information but if we wanted to again if we wanted to go back and try to find this draggable id now we wouldn't be able to do it because we're outside the iframe and again quit your driver once you're done with it okay so in this lecture we went through how you handle iframes or how you work with and around iframes in Selenium the key thing to remember is that you have to specify it and then switch to it if you want to pull data or interact with elements that are inside an iframe within your web page thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on navigating browser history in Selenium Selenium does not allow you to access your browser history so if in your session you're accessing multiple web pages you won't be able to explicitly figure out what they are you can't access that history however what you can do is you can't execute common browser history commands like forward and back which you should be used to if you're used to navigating the web at all in terms of the commands that you use for functionality first you'll open up your driver and your service and all that and then you can get a first URL of interest and then maybe you use driver.get to get a second URL of interest you can use driver.back to go back to that first URL and then you can use driver.forward to move forward to the second URL let's head over to jupiter now to get some practice with this alright so we're in jupiter we've got all of our imports the three websites we're going to use for this lecture that you can import from the settings file are the books to scrape URL the python URL and then this quotes to scrape URL which is just another example or testing website we start up the service, start up the driver so we've got our Chrome window open and now let's get a couple of URLs so that we can get that get that history going so driver.get first let's get the books to scrape URL we'll just go in alphabetical order here driver.get let's get the python URL and then we can get the what is it the quotes to scrape URL so we do all that so we've got a couple websites that are navigating through your history without specifying a URL is pretty easy so we can just use driver.back to go backwards we do driver.back to go back one more time and then you can do driver.forward as well so do that driver.forward will take us to the quotes URL now the one thing that you might try to do or think of doing or wonder if you can do is if you can change these methods together the same way you can do it with findElement and you really can't so if I try to do this I'm going to get an error on that second one so driver.back returns a none so if you try to operate a back on that you'll just get an error so if you want to go back twice what you have to do is specify that on the driver object you can't just chain them together and then quit your driver once you're done with it okay so in this lecture we talked about how you can use the forward and back methods on your web driver to navigate through your web history thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on alerts you've probably seen that webpages can sometimes have alerts or pop-up dialogues that show up as you go through the web page for example you might be submitting a form and an alert might pop up saying are you sure you want to submit things like that in order to interact with these alerts we need to switch to them in Selenium like we switch to tabs or windows or iframes alert objects in Selenium also have methods for dealing with them so we have methods for accepting what the alert says dismissing what the alert says so on and so forth to use this functionality in Selenium once you have your web page of interest and you see an alert pop up you can specify the alert by creating an object so here we instantiate alert the alert variable and we'll set that equal to driver switch to dot alert and that gets us into the alert object on the page then we can use alert dot accept or alert dot dismiss in order to accept or dismiss the alert. Let's head over to our Jupyter notebook to get some practice ok so here we are in Jupyter we've got all of our imports the URL we're going to use is the books to scrape URL so we've got that open in our window and we're going to take a look at alerts here so the first thing that we can do there's not an alert on this page but we can fake one with some javascript again this isn't a javascript course so feel free to remember to use this or not remember we don't expect you to know it's just an easy way of making an alert pop up that we can interact with in Selenium so if you're using Selenium on quote unquote real websites you'll have alerts there that will pop up on their own but we're just forcing one to get practice interacting with them alert function in javascript will force an alert and it should say hey this is an alert and then we can interact with it in Selenium alright so we run that script that javascript and now we've got this alert that pops up on the page so it says books to scrape.com says hey this is an alert alright so that's the alert that we want to interact with to do this in Selenium we can say driver.switch to alert so that will give us the current active alert on the page and we can check that with the text attribute so the text attribute on this alert is hey this is an alert that matches what we see up there so that's all well and good as mentioned in the lecture Selenium gives us some really nice methods to deal with the alerts very easily so instead of having to search for this okay button up here and then press it we can just click alert or we can just run rather we can just run alert.accept and that'll accept that alert message and now we're back to this page so that's one example of accepting it but now what you'll notice is that we can't access the alert anymore because it's gone away so even though we've assigned it to assigned it to this alert variable when we try to pull things off it it's no longer there so we'd have to run it again in order to access that text okay so with the alert javascript function we only get an okay button but there's another javascript function that we're going to use to force up a pop-up box so we want driver.execute script we're going to execute some javascript again and the javascript function of interest to us now is confirm so confirm will give us that yes no type of alert or pop-up that shows up on our page so hey do you want to confirm run that and okay now we've got that alert over here so bookstoscrape.com says hey do you want to confirm okay or cancel so now what I can do is I can switch to that alert just like we did before so driver.switch to alert now I'm in the alert alert.txt check it out hey do you want to confirm that's what we should expect since we wrote the alert and the other useful method for some of these is dismiss so dismiss is the equivalent of clicking cancel on this alert so I do that and you hit cancel and the alert goes away okay so in this lecture we looked at the basically the two methods that you can use to interact with alerts and we looked at how you switch to an alert in order to interact with it in selenium as I mentioned this isn't a javascript course this was just some small amount of javascript to get those alerts to pop up but you'll see them out in the wild anyway and now you have the tools that you can use to interact with them thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on working with cookies and storage items in selenium selenium provides some nice methods for us if we want to interact with cookies on a web page in order to get the cookies we can just use driver dot git cookies on our web driver object and if we want to add a cookie to work with that for some reason we can just use driver dot add cookie and then pass a dictionary with the cookie name and the cookie value unfortunately there's no built-in method for interacting with local or session storage objects with selenium instead if we want to do something with local or session storage what we have to do is rely on executing javascript using the driver dot execute script method the code for working with cookies looks as it's shown here so we can get the url of interest to us then we can use the dot git cookies method in order to return that dictionary of cookies in our session or on that web page and then we can use driver dot add cookie and pass the dictionary that looks like shown here in order to add a cookie to our session so the dictionary will have an item for name and then the cookie name and then an item for value and the cookie value if we want to work with local storage as mentioned we have to use the execute script method to execute javascript so we can write driver dot execute script and if we want to set an item in local storage we can execute the script window dot local storage dot set item and then pass the parameters of a key and a value of interest for that storage object if we want to get something out of local storage we can also use driver dot execute script but instead of using window dot local storage dot set item we can instead use return window dot local storage dot git item and then pass the key of interest to us session storage works much the same way so we can just use driver dot execute script and then have window dot session storage dot set item and pass a key and a value in order to set that item in our session storage we can also use driver dot execute script and then return window dot session storage dot git item passing the key of interest in order to return a specific item out of our session storage just as a note as I've mentioned in previous lectures this is not a javascript course and so we don't expect you to have a knowledge of javascript in order to complete it the javascript parts that we're showing you are more things that are nice to know not need to know it could be that you may have to work with session storage or local storage at some point or you might have to read a selenium code base that uses those things and it's useful to know that you have to use the execute script method along with some javascript to achieve those goals but again it's not crucial to your use or understanding of selenium for now however let's head over to jupiter and get some practice interacting with cookies and storage objects in selenium okay so we've got our jupiter notebook set up and we've got our selenium session open so we've done all of our imports from the settings file we've got the chrome driver path and the website we're going to take a look at is python.org so you can import the python url we've got the service we've got the driver set up and we've got the python url so we're good to go so what we're taking a look at in this lecture is are the cookies and the selenium session storage you can access this type of information in your chrome dev tools dock or window rather if you go to the application tab and under application you've got this storage section so you can look at cookies under cookies so we've got one which is for this website so python.org and these are the cookies that we have so far so we can access these cookies from selenium or from python rather using driver.get cookies and that will give us the list of all of the cookies available to us and if you look at it the names are all the same so we've got this looks like a double underscore utmb double underscore utmz and it's got all this information associated with it I'm not entirely sure what all of this means but the point is you can you can get cookies if you need to interact with cookies in some way in selenium now if you want to set a new cookie what you can do is driver.add cookie so we'll do driver.add cookie and then let's make this a dictionary and so the name of the cookie will be my new cookie and then the value will be let's just make this hey look at my cookie and so for this you do need a dictionary with name and value you can't just make it two things my new cookie and hey look at my cookie that won't work in selenium but we can pass this now using add cookie and so if I refresh this in here in my chrome dev tools tab we've got the new cookie down here so you can see we've got the name let's move this over a little bit the name is my new cookie and the value is hey look at my cookie so that's how we're able to pass cookies to and from selenium using this type of functionality now the next thing that we can do is we can look at local storage and session storage so that's up here you can look at these types of things so you go to the website right now there's nothing there for local storage nor is there anything there for session storage again selenium doesn't provide us with an api to do this type of work or to interact with local and session storage but it's good to know that you can execute javascript if you need to again the use case for this will probably be pretty specific so I doubt you need to remember it to use it in lots of scenarios but it's a good thing to be aware of and so the way we do this the way you set an item is you have to use window dot local storage we'll do local storage first and then do session storage so window dot local storage dot set item and then you have the key and the value so we'll say key one execute that script and now if we want to get the item we can do driver dot execute script and the script we're going to execute is return window dot local storage dot get item so instead of setting we're getting and we're going to search based on that key so it's key one now if I go to the local storage here key one and value one are now set in the local storage for this this selenium session and so if I run this get item here you get value one which is just the value of that variable or that key in our local storage things work the same way for session storage you just change the type of storage so driver dot execute script to window dot session storage instead of local storage that's set item key two and value two and then we'll run that now if we go look at the session storage here for the website we've got the key and the value that we just passed with the execute script method from our jupiter notebook then if we wanted to get that item out of storage we could execute a script again and this time it's basically the same thing so you're going to return window dot session storage dot get item and the key of interest here is key two so that's all well and good we've got our javascript in there execute it and value two which is what we sent to it in our session storage so in this lecture we took a look at interacting with cookies in selenium so it's pretty straight forward because we have apis for that so you can get and add cookies if you want to work with local or session storage however there's no good api for that so you'll have to use execute script the execute script method in order to run these kind of snippets of javascript to interact with those storages okay thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on resizing windows in selenium changing the window size of your web driver session could be useful while you're using selenium if you want to test website accessibility mobile actions and things of that sort some of the useful methods in selenium for getting information about window size and position are as follows you could use driver.getWindowSize to get the size of your window in terms of the height and the width you can also use driver.getWindowPosition to get the position in terms of the x and y position on the screen and then if you use driver.getWindowRect or getWindowRectangle it'll return both the size and the position of your current window if you'd like to change or set the window size and position you can use the following methods set window size just like getWindowSize is a way of setting the window size in terms of height and width of the window you can set the position on the screen in terms of x and y with the setWindowPosition and then setWindowRectangle or setWindowRect takes both of these so you can set both the size and the position if you'd like simpler actions so just minimizing or maximizing your window and maximizing your window is a good thing to do usually at the start of your sessions you can use driver.minimizeWindow to minimize your window and driver.maximizeWindow to maximize it let's head over to jupiter to see these methods in action alright we've got our jupiter notebook open the website we're taking a look at not that it's super important because we're just resizing windows in this lecture but the website we're taking a look at is books of scrape start your service, start your driver get the books of scrape URL and then we can dive into all those methods that we talked about in terms of getting and setting the window size so you can use Selenium to resize your browser window to an arbitrary dimension or position and you can also get information on the current window size so we can get window size for example so it says width of 697 pixels height 775 pixels if we wanted to get the position we could use driver.getWindow position so that tells us the x and the y position of our browser on the screen or our window on the screen and then you can get both of these if you get rectangle which is rectangle and so then you get the height and the width and the x and the y all at once so that's one way of accessing all that information now if you want to minimize your window you can use driver.minimize window run that and go to the background there and then you can use driver.maximize maximize window 2 and now that will open it up and now you can see it takes up the entire screen so let's go back over here to our jupyter notebook now if you want to set window size or position you have all of those so set window so position rectangle size they work the same as the getter methods so let's just do set window rectangle and we can just set in some arbitrary things here so position of 20 y position of 50 let's make the height equal to 600 and then the width is equal to 600 as well run that okay so now you can see we've got the window resize there and that's the way that you can interact with the window sizes in this lecture we talked about all the way you can minimize, maximize your windows or get and set their sizes thanks for joining and we'll see you in the next lecture hello everyone and welcome to our section on filling in forms using selenium and the first thing we're going to talk about is buttons and how you work with them buttons are all over the place on the web in all different types of forms you can find buttons in the web pages you're working on using all of selenium's locator strategies that we've seen in previous sections so for example if you wanted to find all the buttons on a page you could search by tag name and then just use the button tag you can interact with buttons in a few different ways the first is you can click them obviously since that's a pretty normal way of interacting with buttons on a web page and selenium gives us the dot click method for any button element we're working with you can also do what's called context clicking or right clicking on a button or you can do a click and then a hold and a release for things like click and hold or the context click which is a right click instead of having those methods on the web element of interest or in this case the button element of interest what we need to do is we need to use something called action chains a class called action chains in selenium what action chains do is they chain together specific user actions or inputs such as moving a mouse between a specific location clicking mouse buttons for example so that context click where you're right clicking or clicking and holding it pressing keys together so if you want to do something like control P for printing drops and a lot of other different actions we'll see some more of these as we go through this section on filling in forms from the code like we saw in one of the previous slides if you want to work with buttons in selenium like clicking on buttons you do all your imports set up your driver then you can find your element of interest whatever way is easiest and you can click it just using the dot click method in terms of using more complex actions on the button first you have to instantiate an action chains object and the parameter going to pass is your web driver and you should store this as a variable as well to perform actions you can use methods associated with that action chain object so for a context click you can write action dot context click specify the element you want to context click or right click on and then run it dot perform in order to perform the action for click and hold you do basically the same thing except use the click and hold method and then specify the element of interest now that we've talked about how this works let's head over to jupiter and get some practice alright so we're in our jupiter notebook and we've got the imports that we usually have so web driver service and buy but the other thing we're going to want to use because we have to use action chains this time is we need to import that class so from selenium this comes from selenium dot where does it come from webdriver dot common dot action chains and then we can import that action chains action chains class so let's run that so we've got that available the settings from the settings file we've got the chrome driver path books to scrape url we've set up our service and all that so we've got the window open so let's just get that books to scrape url in there go to that page okay so now it's popped open so we're going to be interacting with buttons so the first thing we want to do is select it like anything in any given web page right so let's take a look at this button at the basket so here you've got the button tag it's a submit type the data loading all that stuff but let's just select all the buttons in the page the other thing you might want to be aware of is that with certain types of html markup or classes so with bootstrap for example which is a way of styling your web page you can also get buttons that aren't buttons so for example you could have an anchor tag an a tag with a button class in bootstrap that makes it look like a button even though it's not technically a button html element but that said you can still interact with it the same way so let's get all the buttons on this page so we can use driver.find elements and then we're going to search by tag name because we just want the buttons so by tag name and then we're looking for a button the button tag so that gives us all of the buttons on the page so if we look at that there's 20 buttons 20 buttons on this page which checks out because I think that's the number of books on the page too so we can select the first button if we want so button one is going to be let's make it button zero button zero so we got that we look at button one.text should be add to basket so that's kind of what we're looking at here and if we want to interact with it we can just click on it so buttons have a click method so we can just say click run that and I think this action just reloads the page so you can see all it does is reload the page now what happens however if you try to click on it again is that the page is reloaded so you're going to get an error here this button is not on there stale element reference is the error that you get so we're going to have to find that button again if we want to work with it delete that cell go into here so we can just do what we did again driver dot find elements by tag name button so that's buttons and then button one is going to be buttons zero first item on the list so now we've got that button one and it's just that it's just that add to basket text from the first first button underneath that first book okay so for more complicated actions or click actions with buttons or whatever there is on the page we'll have to use the actions action chains object so we'll instantiate it so actions is going to be action chains then we have to pass the driver so we can just do that and then let's do a context click on that button here so it's in the page so we should be able to see it on element it's the parameter of interest there on element it's going to be that button one so that's just this add to basket here so context click on that which is a right click and then dot perform so you see that now we've got this right click menu up here that we can interact with somehow if we really want to let's click out of that you can also do a click and hold so let's do actions dot click and hold on element again it's the parameter of interest to us and we just want to do that on button one and then we'll end with perform so it performs the action so you can see that here it's kind of chains it's got that black outline and then if we wanted to I think we could do actions dot release dot perform and that should unclick the button right so now the page is reloaded again and like I mentioned you can do some of these actions with various parts of the page so if we go up here the the books to scrape is let's click on that it's an a tag right we can still click on it we can still act like it's a button and click on it so the link element let's just do driver dot find element find element a by tag name so we're just going to get that first link and then we can click on that so link element dot text is going to be books to scrape so again it's the website title up here books to scrape and we can click on that because we still have access to that click method and now it's just reloaded the page it's just gone to the home page again ok so in this lecture we learned how to interact with buttons on a page and we got our first exposure to that action chains class again so the action chains will become useful in the future as well as we deal with things like drag and drop sliders and sending keys to a web page thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture talking with input elements or text boxes in selenium input elements or text boxes are very very common on forms throughout the web in terms of the actions you'll usually want to be performing on these fields you'll probably want to fill it out with some text or clear it to clear it you can select your input element or text box and then use the dot clear method on it that will remove any text inside the text box in order to fill it out you can use input element dot send keys and then pass the string of interest for how you want to fill out the field let's say you've got a first name field that you want to fill out you can get that element and then you send keys and pass the string with the first name of interest that you will be using to fill out that form field in terms of the code that's in service and drivers setup you can get the input element of interest using any of the methods we've used before so id, tag name, name any of that stuff then you can clear it if there's any placeholder text or if you want to make sure that it's clear when you send keys so you can use dot clear and then you can send the text using dot send keys with the string of interest so here we've got filling forms is cool and that's the text that we're sending to the input element now that we've talked about how this works in Selenium let's go over to jupiter to get some practice alright so we've got our jupiter session open our jupiter notebook open we don't need that action chains import that we had in the last lecture so that's been removed from our imports we just have web driver service and buy then from the settings file we're going to be taking a look at the python web page so like you can see over here we're on python.org we've got our service and our driver setup and then we've gotten that url so now the input element that we're going to take a look at the obvious one on the page is this search box here so where we can enter some text and then go search for it on the website so if I inspect that with my chrome dev tools inspectors what you get is this input tag here it's got an id of id search field and a name of q so we can search for that either by id or name let's use let's use the name for now just because we haven't used that in a while so what we want is we want to input element call this input element and this is equal to driver dot find element and it's going to be by the name and the name is q so now we should have that input element that's a search box if we dot let's check the tag name so the tag name is an input so that's good and then let's check the attribute so input element dot get attribute and the attribute let's take a look at the placeholder it will be search so all that works like with locating elements like we've seen before in terms of the text in this we can check that with the text attribute and there's nothing in there so the search we see like we saw up here that's just a placeholder in the html it's not actual text in the text box so if we wanted to uh maybe send it some text to do a search I'm going to make this a constant so that we can reuse it so let's do search text python news it's going to be what we're searching for so we've got the input element we can use that and then what we want to do to fill it out is we can send keys and we're going to send it that string that search text string now you can see it's been filled out up here right we've got python news if I scroll back over to the right you can see the beginning of it we've got the end of it that's the string that we're going to be searching for now if I wanted to clear it I could just do input element dot clear and now it goes away now we're back to just that search placeholder there alright so now if we wanted to run a search what we can do is we need to uh we can let's get that search button so search button and that's the this one here so let's click on that let's go so it's a button it's got a name of submit id of submit so we can use either one of those to find the button so let's do driver dot find element let's do id id is submit so that's pretty easy the text on that let's just make sure is go so that matches up so now we can send the um send that search text search text that's right dot send keys so input element dot send keys and then I can send it search text let's do that so that's there and then we can do search uh let's use search button and we can click that button like we saw in the previous lecture it's just a button we can interact with it that way so now it's gone the search has been performed uh scroll down a little bit so now you have all these results here on the python website alright so in this lecture we took a look at how you interact with text boxes so it's pretty there's a couple of easy steps so basically you want to find the element you want to clear it if there's stuff in it using the clear method and then you can send keys to send a string of interest and if you want to click that go button you can find the button and click it like we saw in the previous lecture as well okay thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on interacting with radio buttons and checkboxes in selenium radio buttons and checkboxes are used to specify options in html forms both radio buttons and checkboxes work fairly similarly to what we've already seen with buttons in selenium you can use a click method on them for example to click the element and select it and you can use the is selected method to check to see if the element is selected in the form the one difference to be aware of with checkboxes and radio buttons is that if you click and then click again on a checkbox that will select and then deselect it with the radio button as soon as you click it it will be selected and it can only be unselected if you click on another radio button element to use this in python you can do all your imports and service and driver setup finally you can get the checkbox element that you're looking for or that you want to interact with you can check to see if it's selected if you want to make sure that it is or isn't before you interact with it the dot is selected method and then you can select it via a click so the dot click method as I mentioned radio buttons work much the same way with the one exception that you can't unselect it by clicking it let's head over to jupiter now to get some practice with this here we are in our jupiter notebook we've got our imports the website we're going to take a look at in this lecture is the goldbugs website so this is just a fake band website basically you can see it's open here so the goldbugs.com or you can import it from our settings file set up the service, set up the driver get the url and then we're good to go so in this lecture we're interacting with checkboxes so if you scroll down here you can see there's a form here so let's click on this to see if we can find the form or find the generic object here so we've got the div and I think this div holds the form so we've got the div with the form wrapper form inner wrapper and then we've got this form here that contains all the stuff that we need so let's get that let's get that form or the other thing we can take a look at actually is the is here so we've got the field set so let's click on this where is this okay we've got legend and actually here we go we've got the field set so it's actually a field set tag that we're interested in and it's got an id so let's control c to copy that and just copy it out of the dev tools and then we can select that so that we can look at the checkboxes there so field set let's call that field set let's pull this up a little bit field set is driver.find element by id and then copy and paste that id so it's fairly long you don't want to type it out so okay shift enter now we've got that field set tag name let's just check this out so field set okay so that's kind of what we expect so now if we want to pull the checkbox elements out of that we can just look inside the field set we don't have to look for the specific thing in the entire html this is that chaining of locators right so we've got these option divs here with a label and then you've got an input so the input is over here where you can click on it with the element explorer here so it's input type checkbox so that's probably one way that we could look for it so let's do field set dot we're looking for here find elements so we want all of the checkboxes on this so we can do let's do by xpath we're looking for the input maybe make this on a new line we're looking for the input and an input with a certain type so we want the type to be checkbox and you can check this in the in the depth tools console tab if you like but it should work so we're not going to check it this time we can just it works so it's a good xpath so we got the checkboxes so checkbox one just make that checkboxes checkboxes let's see one and so checkbox one let's get attribute of the value so the first one here if we click on that is no response required so let's get attribute value no response required and so if you look at zero that text it's sort of like this hidden checkbox on the page so we don't have to worry about that that's why we're not using zero we're using one instead okay so get attribute no response required and now we can check is it selected so it's not selected on the page let's just verify that is selected false so it's not selected so if we want to select it we can click it checkbox one click now it's selected you see we got the checkmark that shows up on the page there we can check if it's selected again so click or not click is selected and now it's true right we switch the state on that checkbox if we do it again checkbox one click and then check it's gone back to not being selected so false now for the next part we can take a look at the radio buttons so we'll just find elements on the page if I click on this let's click on that input so it's an input of type radio so that's what we're going to look for and we're going to use an X path again for this so radio buttons on the page is equal to driver dot find elements we're going to go by X path again and then the string we want is pretty similar to what we did above so slash slash input so relative inputs with a type field that's equal to radio and that's what we're going to search for here so run that radio length of radio buttons seven and the reason for that we've got these two here and then we've got these five radio buttons down here to work with so we're just going to probably deal with the type of event radio buttons so let's do radio button one is going to be equal to radio buttons zero so the first one which should be private show so if we check that out radio button one dot get attribute and then we want to look at the value so the value like we can see here is private show or the text would be private show too I think radio button one get attribute value private show that's what we expect we can check to see if it's selected and this is false so we can click it just like we did with the with the check box right but now it's selected so radio button one is selected is true unlike with the check box like I mentioned before if you click it again it's not going to do anything it's still selected right so that's just one thing to keep in mind while you're working with radio buttons versus working with check boxes here okay so in this lecture we learned how to click and unclick check boxes and how to click radio buttons as well and then we talked about how to unclick radio buttons essentially to unclick them you have to click another one to unselect it and we went through the methods that you can use in Selenium to use those actions thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on working with drop down menus drop down menus are really select elements or select tags in the html source file of the web page that you're interacting with the options in these drop down menus or select elements are contained within the option tags you can interact with these elements using all the basic methods that we've learned so far for forms so you can select them using whatever by method that you that you choose or whichever one is easiest you can click on that if you'd like or and this is really the much easier and the better way of interacting with them you can use what's called a select object select objects in Selenium are basically objects that provide very nice wrappers and very nice methods for dealing with these select or drop down menu elements on a web page for example you have the options attribute of the select object so you can see what the options are for the things that you can select in that drop down menu you can then select by index so 0, 1, 2, 3 so on and so forth you can select by visible text so that's going to be the text that's visible or you can select by value which is the HTML attribute associated with each option these types of methods and attributes also work with multi-select menus where you can select more than one thing at once additionally with the multi-select menus we get some deselect methods as well because you can both select and unselect multiple items in the code the way you work with these things is you have to have an import from a module that we haven't used before so from selenium.webdriver.support.ui you can import the select class then to instantiate that select object you find the select element on the page or that drop down element and then you pass it to the select object to instantiate it to look at the options on that object you can write select.options and that will list all the options you have for the drop down menu in terms of selecting items on that list in order to fill out the drop down menu in a form you can use the dot select by index method and then just pass a number of the index that you want to select in order to select by value and so that's the value of the HTML attribute not what's shown on the screen use dot select by value and then pass that value string for multi selects as I mentioned you also have deselect methods that are equivalent to their select method or the opposite counterpart rather of their select method so you can have that select object and then deselect by index works just the same way except you're deselecting and then you've also got deselect all so that's if you want to clear out a multi select and start again let's head over to jupiter now to see how these work on the web okay so we're in our jupiter notebook and we've got all of our normal imports the one other thing that we're going to want to import for this lecture because we're going to be dealing with drop down menus or select items from a selenium perspective at least is we want to import from selenium web driver dot support dot UI we want to import that select class so let's run that and then we're good to go we're on the goldbugs website so we've already done our driver dot get and now we can take a look at this so if you look at this website we can scroll down to the booking form and down here we've got a how did you hear about us drop down menu so let's click on that for a second and take a look at this select item so there's a few things we can do here to interact with it so if we want to just select it the kind of our classic methods we can select the let's do a select call it select element new driver dot find element I think the best way of doing this is probably by ID so let's do ID and then we can just copy that from the ID field in our dev tools copy that paste it over here it's a little bit long but that's okay and then we can run this and then we've got our select element there so now let's print the let's do select element dot text maybe see what's inside so then we've got kind of this messed up text that's all the that's all the options for it or maybe if we do print then we can get the get the nice way of formatting it let's do that so here we've got all the options for that select element the other way that you could get the options is that you can maybe do option elements let's do select element because they'll be inside there find elements we'll search by tag name and so if you look inside here we've got these option tags right scroll up a little bit so option, option, option with some values so let's just do by tag name we want the option tag lower case okay now let's loop through this real quick just to look at some stuff so for the element in the that list of option elements we can print an F string and what we want let's do element dot text and then do element dot get attribute and we'll do the value there so the value and the option like you can see in the html here that's like google facebook that's in the html that's not in the user facing text on the page let's get this to stop scrolling there so let's run that so here they're one and the same but they might be different on a different web page so google is a text google is the value attribute so that's one way of getting it and in terms of clicking on it again we'll take a look at the select but we're doing it the hard way now so let's do option elements let's do hit the bracket there let's hit the right key so let's just do the last one negative one which i think is other and we just click that elements so let's try that so now you can see we've got that clicked and it's clicked to other if we run this again but let's do it with one then it should switch to you can see it switched to facebook which is that zero one it's that second option one in python because it's zero based indexing of course so we've gone through this that's one way of dealing with it but it's really not the easiest way you want to make your life easy in selenium and so what you can do instead is really instantiate a select object to deal with these drop down menus we've already got the select element picked out on the page so we'll just do the select object in this way so now if we want to look at the options we can just do select options and then we've just got essentially a list of these web driver elements so instead of doing that let's do let's do a print and we'll do a list comprehension so that's a little bit easier to read let's do option dot text for option scroll this down a little bit for you guys for option in select dot options so select dot options is that list of web driver elements that we see down here and so we're just going to get the text off of each one of those options and now you can see we've got Google, Facebook, Instagram so this is a lot easier than trying to figure out where these options are on the page and if we want to select we can just select by index as well so select by index we can do index let's see 0, 1, 2, 3 so if we do 3 that'll give us other so you can see now this is switched to other right another way of doing this is select dot select by visible text in this case visible text and value are the same so it didn't really matter but sometimes value and visible text are different so you might want to use one or the other but here we'll do visible text and let's do Google so now we switched the option to Google so as you can probably see by this this is a lot easier way of dealing with drop down menus in Selenium than having to go through and find element by element basically now if we were to do a multi-select you'd have these methods like deselect all or deselect by visible text so on and so forth this isn't a multi-select so we can't do that but just so you're aware in case you see multi-select forms on or multi-select fields on forms you're dealing with in Selenium ok thanks everybody for joining this lecture where we saw how to deal with drop down menus using the select class in Selenium and we'll see you in the next lecture hello everyone and welcome to our lecture on interacting with calendar pickers in Selenium calendar pickers have really become a very common form element in a lot of different websites these days so you might think of an airline booking site for example where you have to pick the date of your flight to a place and the date of your return flight you could think of a hotel booking site or a hotel reservation site where you have to put in the start date and the end date of your stay all of those might be involved with calendar pickers calendar pickers are generally composed of three components of interest to us in terms of how we use Selenium the first is an input field where the date of the list will end up the second part of it is usually some javascript or jQuery that will make the calendar pop up as soon as that input field is activated and then the calendar itself is usually some sort of table element to hold the rows and columns of the dates of any given month these components and our list of these components really provide us the outline of how we're going to interact with calendar pickers in Selenium the first things first so we have to find that calendar input element on a page then we can activate the javascript or the jQuery that will make the calendar pop up so we can do that usually with a click in order to activate that input element and then activate the javascript once the calendar picker has been activated and we've got that table element showing on the page you can select the date of interest column or using some sort of data coming from the html tags in that table sometimes you can also use send keys on the input element to simply fill out the date rather than picking it from a calendar picker the one thing to note about this method is that you may still end up activating the calendar and so then you'll still have to click into the text box or click the date in order to get the calendar to go away now that we've talked about the steps of how we interact with a calendar picker in Selenium let's head over to our Jupyter Notebook to get some practice okay so we've got our Jupyter Notebook open we've done our imports the website we're going to be using is the jQuery UI website so you can import that URL the jQuery UI URL and then once you have your driver set up the URL you can actually get is the date picker so we're going to be interacting with that jQuery functionality here to get some practice with using date pickers in Selenium the one thing to note on this though if you click on this so let's click on this date picker here this is inside a head and a body inside an html inside of an iframe with a class of demo frame so as we should remember from our iframe lesson we first have to switch into that iframe before we can start interacting with the date picker if we try to just do it the normal way we won't be able to do that because that's how iframes work so we can do iframe element is driver.find element by css selector and then it's going to be iframe.demoframe so iframe tag with a demo frame class then we can do driver.switchTo we're going to switch to a frame and the frame we're switching to is that iframe element so now we're inside of the iframe and we can try to find that input element that date picker so input element is going to be driver.find find element let's take a click on this again let's just see it's got an id of date picker so we'll just use that that's the easiest way to do it so find element by id date picker that's the id so now we've got that input element if we want to activate it we can click on it so now you can see we've got this calendar that's popped up right that we can use as a calendar picker so selecting an element in here is basically the same as selecting any element that we might want so if I click on that so you've got this a and you've got a td which is a table cell basically and it's got some information so data month data year and all sorts of other stuff in here a data date date 18 so that's today's date so you've got the month you've got the year and then you've got the data date now this is slightly more complicated so in order to select things here you can do a couple different things but I actually think the best way is probably to do an x-path and I'm just going to select today's date because obviously you can watch this video at basically any point in time here just to select a relative date rather than picking something specific so that everybody has to change things so the first thing I'm going to do is import let's do a datetime import so from datetime import datetime just so that we can get the now the now method so let's make a couple of objects we want month day so now is datetime.now the year is going to be now.year month is going to be now.month and then day is going to be now.day so run that if you look at year 2022 you look at month month is 11 and then day is 18 so that's everything that we really want there now we need to do the x-path for this so you can test it out in the chrome dev tools if you want but we're just going to construct it so the first thing we need is we're trying to get to that we're trying to get this a here that's got a data date of 18 but first we have to go to this td with the data month 10 and the data year 2022 so how do we construct this we can do an f-string so we'll use those variables that we just made it's relative so we want to go to the td first td where let's see we've got data month and data year those are the two variables that we want to match or the two attributes that we want to match so here if you look at the month the month was 11 but in here the data month is 10 so again I think that's a one indexing thing so you want month minus one to match data month and then data year is just going to be year so that's the two parts of that and then the next part we want once we have that td we have to go inside it to the a tag and we want the a tag where data date is equal to our day so this is a long f-string but it should make sense to you once you compare it to here so we've got year month day it's going to be 10, 22 and 18 so we're going to bring this down into this a tag that we want to click on or that box that we want to click on in the date picker now to find that element, let's find that let's call it date element driver.findElement we're going to search we've got the x path there let's just search by the date path so now we've got it selected should be selected let's see could be that I've got a bad string somewhere invalid selector okay I'm missing a missing a quote here I think yeah I'm missing a quote and okay I've got the brackets messed up so let's see delete that close the bracket so that's one and then we've got this other bracket over here so let's try that okay there we go make sure to check your brackets and your quotes on these things so if I look at the date element.text that should be 18 that's the date we want to select so that's all good and then we can just click it let's click this element and then it gets filled in so you see now we've got this date in the input field there 11, 18, 22 that's today's date selected since it is an input field you can still clear it so let's do input element which is that text box there clear so that'll clear out the date but you can see it also activates the the date picker so now if I've got this input element and I want to try to fill out the date using using keys so we got month you saw the format it was month, day year so we can send that you can see we filled it out now and it's valid but we still have this date picker up so if you fill out your date this way if you just send the keys you're going to have to click on it or something to deactivate that javascript with the date picker click on it or if I maybe click outside of it again you know then that'll be what this activates the the calendar table in there and we'll clear that off your screen so like you can see there's a couple pros and cons to using both methods if you're clicking on the input field and then selecting from the calendar table that takes a little while to get your search string that you're looking for to click on the right date but if you send the keys to the input then you kind of have to click around to get the calendar to go away because sending keys to the input will usually activate the calendar okay so in this lecture we went through how you interact with a calendar picker with selenium thanks for joining and we'll see you in the next lecture hello everyone and welcome to the lecture on uploading and downloading files in selenium generally speaking file uploads only involve sending text or keys to an input element like we've seen with any text box or input field before however one thing to note is that the path sent to the input element must be a real path on your computer and it must satisfy any upload restrictions so for example if the form only accepts .png files you should only send a .png file if you don't adhere to these rules so if you don't send a real path or if you send a file that's not permitted by the input element you'll trigger an error in your python code in terms of file downloads they're pretty straightforward all you have to do is click on the element or the download button of interest and that will activate the download and it will end up in your download directory one thing to make note of however is that you can change the download directory with chrome options the way this looks like in code is that you can instantiate an options object using webdriver.chrome options and there are other ones for firefox or the other browsers then what you can do at least in chrome is you can do options.add experimental options the first parameter is going to be prefs for preferences and then you can pass a dictionary with these preferences for us right now we're only interested in changing the default directory for our download so we'll pass a dictionary that says download.default directory and then the value for that key is going to be the path to your default directory if you want to change it then we can instantiate our webdriver which is normal just using webdriver.chrome passing the service but then we'll pass another parameter which is the options parameter so we can just pass that options object that we defined above if you don't do this before starting your webdriver the options you add won't get added to your webdriver so you have to make sure to do it before you start your webdriver not afterwards let's head over to our jupiter notebook okay so we've got our jupiter notebook open with all of our imports the websites we're going to be looking at for this practice are you can import them from the settings file it's a python download url so python.org slash downloads which is open to the left there in our chrome browser and then the imager upload url as well we'll take a look at that later when we talk about file uploads start the service start the webdriver and then get that download url and then we can get going so to download a file there's typically a button or something similar a link or something that you can click on in order to download the file of interest maybe it's in a form maybe it's not in a form so here it's not really in a form but we can still get some practice with it let me just zoom in on the html here we've got this button to download python 3.11 and if you look at the class there's a p here with a class of download buttons and then an a tag with a class of button but there's not really a good id or a name to take on that's very specific there might be other things in the page with a button class for example so what we can do is we can copy this xpath and search for this button element by so I'll just store this as a string constant so button xpath and then paste the xpath that we copied from our chrome dev tools so you can see it's pretty long which is why we don't want to type it out by hand we can just copy and paste it from the chrome dev tools that's a lot easier it's a much better way of doing it then we can find this download element so download element we'll just do driver.findElement we're searching by xpath here and then our xpath is that string that we just wrote so the button xpath okay great so now we've got that going to so to download something all we have to do is click on it so it might be a button it might be a link with a button style but we can just click on it and that will start the download so you can see here on chrome there's a little bit of a warning so we can keep it there we're not worried about it it's just a python.exe file and then the download starts and once it's done you'll have your whatever file it is that you downloaded the one other thing that we do want to talk about though for you to be aware of is that chrome options that we were talking about in the lecture so by default and most browsers your default download folder is just your downloads folder right so on windows for me it's users b-a-b-i-g downloads and the file will download into that folder if I want to change that however I can change that using chrome options so let's cancel this download for now cancel that I can do that with chrome options so the download directory for me write this out this is windows unfortunately on windows you have some issues with the backslash so I'm going to do that without the raw string just so that you see the error so I'm going to make this c users b-a-b-i-g and then desktop so download to the desktop instead of instead of the downloads folder if I do this I'll get an error because of these backslashes so you have to double slash out of it or pass it as a raw string on windows you usually don't have that problem on linux or max because there are forward slashes for the file paths and for this one just as a notice if you are on windows make sure to end it with the double slashes because if you don't changing this option it won't work I ran into that issue when I was testing this out so our download directory is just that string that we've changed here and then if I want to change it in our in our web driver what I can do is I can instantiate this chrome options object so I'm going to say webdriver chrome options that's going to be our options variable and then one of the things you can do is you can add an experimental option and this is a way of changing preferences or changing certain setup items or options in chrome so what I'm going to do here let's get this just delete this make this a little bit easier to look at here okay there we go the experimental option we're going to add is to prefs so that's preferences and then what I can do is have this dictionary the option that I'm going to change is download default directory so that's that default download directory and what I want it to be is that string that I just wrote so it's going to be my desktop my desktop folder in order for this change to take effect you do have to quit your driver and start it up again you have to pass it as an option first thing so we're going to start this up again our driver is webdriver.chrome service is the service object that we have and then the other variable that we're going to pass now is the options variable or the options parameter right the options variable that we set up that's that chrome options object with the default directory changed so now if I start that up the window will pop up then we can do the driver.get let's do the python download url just like before so that'll take us let's go back click on something that'll take us here and then we can find that the download element is driver.findElement searching by xpath and then we've got that button xpath string and then you can click on that downloadElement.click and the download will start again and now if I show this in the folder the file path is my desktop right so that's the changed file path and the other way you can check that in chrome is by going to your settings and then clicking on downloads and that'll tell you what the default download location is so that's essentially what we're doing with that option is we're running that change there okay alright so we talked about talked about downloading files using selenium now let's talk about uploading files so in order for an upload let's first get our upload url so that's imager and so we've got this here and what we're gonna do is we need to choose the photo or the video so let's stop this cancel that download exit out of that to maybe get some more space maximize the window here and then inspect so inspect this element dock it to the bottom zoom in okay great now let's click on this choose photo or video so if I click on that it's label for input we've got an input here so it's got an idea of file input that's good we can use that you can do multiple ones we're just essentially all we have to do is send keys to that element so let's get the upload we'll call it upload element this is equal to driver.find element and then we're gonna look by id because it has that id and id is file input right so I can just copy and paste that into the as a string into that driver.find element so now I've got the upload element let's just check that it's got tag name so it's an input alright so now we can just send keys really in order to fill that out to upload a file but what you'll find is that you get an error if you send a path that doesn't exist so upload element let's send keys and I'm gonna go find let's see I've got this intraslide png file properties so it's users be a big desktop and then intraslide.png so I'm gonna the first thing I'm gonna do let's do this remember for double backslash for windows let's do fake image.png if I send an image that doesn't exist I'll get an error so if I do that it's gonna say blah blah blah invalid argument file not found right fake image that doesn't exist so just to remember if you are uploading a file you have to make it a a real file that really exists otherwise you'll get an error now let's say intraslide this exists so we send that and the file gets uploaded so now we're uploading that image file to imager here we go everything's good and we've got everything that we need okay so in this lecture I realized it was a long one but we talked about how to download items and how to upload items and it's really just interacting with elements almost the same way we've seen before the one the couple of differences are we talked about how to change your download directory using chrome options and then we talked about the error to be aware of so that's passing a string to a file that doesn't exist so if you are uploading uploading something you need to make sure that file actually exists thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on dragging and dropping items in selenium since what drag and drops really are are essentially mouse actions and they really involve a couple different actions so you'll click, you'll hold you'll move and then you'll release we'll need to go back to our action chains object again we can use a pure drag and drop with a source and target element in order to drag and drop items in selenium what we can also do however is we can use a drag and drop with an offset so we'll start at a source element on the web page and then drag and drop that source element with some sort of x and y offset what this looks like in code is that we have to instantiate another action chains object so we can just make it actions is equal to action chains and then pass the driver as the main parameter if we're doing a drag and drop with a source and a target element we can end those two elements on the page using whatever method is best for finding those elements and then we can pass those elements as parameters in our action chain so we can do actions drag and drop then we can pass a source element and a target element and then we can end it with dot perform in order to perform that action drag and drop with an offset works much the same way except you don't have a target element you just have the source element and then an x and y offset let's head over to our jupyter notebook now to see how this works in practice alright so we're in our jupyter notebook we've got our imports from selenium the one thing to note is that the other import that you're going to need for this lecture is from seleniumwebdriver.com and action chains you need to remember to import the action chains class because we need that for the drag and drop action the URL that we're going to use is the jQuery URL so you can import that or the jQuery UI URL rather you can set it up and then we're going to be going to the jQueryUI.com slash dropable so that we can work on the drag and drop so this is the drag and drop you can play around with it in the page if you want to find this right now so it's a div in there so it is an object with an id but if you look at it as with many things on the jQuery UI website when you look at it it's inside this iframe with a class of demo frame so first we have to go inside of that iframe in order to interact with the drag and drop object so the iframe element let's find that we can use driver.findElement we're looking by cssSelector and it's going to be an iframe tag with a class of demo frame so find that we've got the iframe and then we can use driver.switchTo .frame to switch into that iframe element so now we're inside the iframe if we want to do a drag and drop we have to instantiate actionChains object so action is equal to actionChains and our driver parameter is just going to be our web driver so do that now for the drag and drops you can do this two ways like we talked about you can do it with a source and a target which is what we're going to do first or you can do it with an offset so source and target we need to find the the source and the target so the source is going to be this drag element and the target is going to be the drop element so if I click on that we want this div with an id of dragable and then the target is this div with an id of dropable so the source element is equal to driver.findElement we're looking by id because we have those nice id's to use so the id is dragable that's the source element the target element is driver.findElement again we're searching by id and this id is dropable so like you see down here in the elements tab of chrome tools or chrome dev tools div with an id of dropable is where we want to drag and drop that dragable element so now we can set up the action to drag and drop things so we can say action we're going to drag and drop first we've got a source element so source is this source element and then the target element so the target is the target element and then we can perform that action within the action change you always want to end with perform so that the action gets performed so let's run that and that's what you see here so now we've dragged that square from over here where it was over on top of this this drop target so it says drop right the state of it changed because we dragged the square onto the target now if we want to do it with an offset so an x and y offset what we can do is we can do the same thing action.drag and drop but we're going to drag and drop by offset we still need that source so the source is going to be the source element but then we have the offset so x offset is the parameter but you can also just pass it positionally if you'd like and then the y offset let's just make that zero and then as always you need to end with perform so that the action gets performed so if I run that you'll see that square has moved a little bit not too much but a little bit and that's how you do the drag and drops so in this lecture we went through how you drag and drop elements using selenium and the way you have to do that is with action chains and then within action chains there's these two methods so drag and drop where you have source element and target element or the drag and drop by offset where you have a source element and then some x and y offset source element around thanks for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on interacting with sliders sliders are an interesting form item or form element because they can really be interacted with in two ways in selenium the first is you can just click on them as an element if you do that what will happen is that you'll move the part of the slider that's the indicator for where you are on the slider that will just go to the center of the element really the other way is you can use action chains again within action chains you can use the following methods to interact with that slider you can move to an element with an offset or you can drag and drop by an offset in terms of what this looks like in the code if you're just clicking on a slider element you can first find the element to select it so here we're just doing slider element is find element and then find it whatever way works best and then you can just click on it with dot click again this will click on the middle of that slider element the other way is you can use action chains so we instantiate that action chains object with the driver as our parameter and then we can move to an element with an offset so we specify that slider element then we can specify the x and the y offset click on that offset from the slider element and then finally dot perform to perform that entire chain of actions this essentially moves you to the slider element of interest and then moves you some direction in the x and y direction in order to click on that slider additionally you can use a drag and drop so you can do action dot drag and drop by offset specify the slider element and then specify the x and y where you want to drag and drop that indicator on the slider finally you end with dot perform so that the action gets performed via the action chains object let's head over to our Jupyter Notebook now to get some practice with this functionality alright so we're inside of our Jupyter Notebook like we did in the last lecture you need to remember all the imports and then also remember the import dot webdriver dot common dot action chains import the action chains class because we'll need that to interact with the slider here from our settings file the URL we're going to use is a jQuery UI URL again set up your service and your driver and then the URL that you're going to get is the jQuery ui URL plus slider so we're going to an example jQuery slider on this page as always with the jQuery website we're going to interact with the slider so if we click on it it's got this slider class it's got a div with an idea of slider so that's what we're interested in but it's still inside this demo frame, iframe so we need to move inside there first so again the iframe element let's scroll this up a little bit iframe element is driver dot find element we're looking by the CSS selector and it's an iframe tag with a class of demo frame select that and then we can switch into it so driver dot switch to frame and then we'll just use that iframe element great now we're inside that iframe so the slider element that we want we can click on it again just to take a look so we're going to do driver dot find element and it's got an id so let's click on this again so that we remind ourselves what the id is so div with an id of slider it's right there so that's what we want so searching by id and then the id is slider alright so we've got that slider element selected one option like we talked about in the lecture is to just click on a slider and that will drag the handle so if I do slider element dot click it basically just clicks in the middle there so you can see the box was over here and now it's in the middle that's basically what happens when you click on a slider object it'll just move it to the center of the element the other option that you have is to move with an x and y offset or to drag and drop it basically so to do that we need to instantiate the action chains object so we'll say action chain action chains rather the driver parameter is our driver set that up actions action chains no s so there we go so action is the action chains object we can move you can do this move to element with offset is the one that we want so the slider element is what we're going to start with so that's basically where your mouse is starting and then you have an x and a y offset so we'll go 40 in the x direction 0 in the y direction and then we'll click so we'll click that and then we can perform so that's going to combine all of these actions in the action chain so first we move to the element with offset so it's like we move to this slider and then we move to the right since it's a positive x value and then we click so that's like clicking the slider that's kind of like doing this so now I'm clicking around with the slider that's basically what we're doing with this set of action chains so move to element with offset click and then perform that so if we click on that now you'll see it's moved that box has been moved over to the right because we gave it a positive a positive x value so we can try that again with a negative x value just so that you get an idea of how things move so move to element with offset so slider element we're going to go there then let's go negative I don't know, negative 100 so we'll go to the left here and then zero on the y click and dot perform so again it's kind of moving down to this slider element moving over to the left and then clicking that's the action that we want to do here so now you can see it's moved over to the left so that's a second option for how to interact with sliders the next one is that you can do a drag and drop so we can do action drag and drop again by offset it's probably the best way to do this so we start the slider element let's go over to the right again let's make it, I don't know, 150 zero on the y because we don't want to go up or down really and then just perform that so if I run that then I'll move it over to the right again so you see it was over here now it's moved over to the right if you have some sort of slider where you need this value you're going to have to interact with that web page to figure out how exactly to get that exact value that you might need for changing a slider on whatever web page you're interacting with but in general these are the three ways that you can do a drag and drop with a slider or the three ways that you can interact with a slider so you can click it, that'll just send it to the middle you can do a move to element with offset and then just remember the positive and negative X values or you can do the drag and drop with offset which works much the same way okay thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on keyboard actions in Selenium as mentioned before in previous lectures the action chains class enables us to automate certain basic user interactions in the browser this can include running key combinations that may be used as shortcuts on certain websites one example of this that you might be able to think of is the keyboard shortcuts that we can use in Gmail for example or Outlook or any other type of email website or email application if we want to use this type of key combination functionality in Selenium with special keys like escape or control or alt we'll have to import a new class which is the keys class we need to use action chains to send keys to the browser as opposed to a specific element and to mimic those key combos so control P to print we can't just use send keys and then try to send the keys control and P to some sort of element on the page that won't work we have to use the action chains object we can use the methods key down key up and release action chains class in order to mimic those key combinations what this looks like in code so first we have to import that new keys class that we talked about so from Selenium webdriver.com and keys you can import the keys class you instantiate the action chains object like always passing the driver as that main parameter and if you wanted to emulate control D for example there's usually bookmarking in most browsers you could do action dot key down send keys dot control so that's the way you send that special control key then you can also do dot send keys and pass the D key as a string then you can do release and then you can run perform so that's the way of emulating that control D action that you might use to import that new keys class that we talked about in our bookmarker page let's head over to our jupiter notebook now to see how this works in practice alright here we are in our jupiter notebook we've got a few new imports are a few more imports than usual for this section so we've got our web driver our service make sure to import action chains that we talked about in the lecture is from selenium dot web driver dot common dot keys we need to import that keys class because that's where we'll get the special keys on our keyboard to use them so things like control tab escape all that good stuff the URL we're going to be using for this lecture is the selenium URL so you can import that from the settings file set up the service in the driver and then get that selenium URL and if I go over to my chrome browser here one of the things you might see on this selenium page is that we've got this search box and the search box actually has an image to show us the hot keys for it which is control K so if I want to do a search here I can just open that up so I just typed in control K on my own not from selenium so if I want to interact with that we can do that with selenium so first things first let's find this element so I'll click on it click on this search bar here and really what am I looking for I think I'm looking for the span actually so we can just do the span send it to this span here so we'll find that element so the element search element let's do search element and actually I think in this case because you can send it to the page we just need an element on the page so search element is driver.findElement we're going to look by CSSSelector and what we're going to do here what do we want we want a span with a class let's say doc search button container doc search button container ok so we found that now if you want to you can't just do you can't just send keys to the element using our send keys method that we've talked about before so we want to do control K so we can do keys.control let's try plus K you might think to do that that's just going to get an error though you can't interact with the element in that way so it'll throw this error so that's not the way that we want to do this kind of hot keys stuff in selenium in order to do it we have to use an action chain so we'll say action is action chains driver is equal to driver instantiate that so one of the ways of testing that search shortcut is we can do action key down we want to hold down the control key so keys.control then we want to send keys we want K so again remember control plus K is what's the is the search shortcut for the selenium website then we want to release both of those keys on our keyboard and we want the action to be performed in selenium so we have to end it with dot perform so this combines a lot of different things but it's pretty straightforward and it's what you do when you interact with the browser you hold down your control key you hit the K key and then you let go of all the keys so that's what's going on we can run that and now you can see in our browser we've opened up the the search box here so we've successfully used a keyboard shortcut in selenium just by using those action chains now here we've got an input element so again we can interact with it if we'd want to it's id is doc search input so let's use that so what am I looking for input element is equal to driver driver dot find element we're going to search by id and it's doc search input so now we can search with this and the reason we'll search for this or search with this is that this also has some hot keys that we can practice so we can send up and down we can send escape to close but right now there's nothing in there so I want to get some search results in there so we've got the input element and now for this one we can send keys directly to it because it's just a normal box right? so let's look for some information on web driver weird let's try that again I don't know why it sent that element first so now we send keys web driver so that's in the search box and then we could scroll through the results if we wanted to action change so we've got our action object let's send some keys and again we're going to use the keys class because these are special ones let's do arrow down remember to close it with a perform so that the action gets performed and now we can go down the page right? if we're looking through these results so keys dot maybe arrow down again perform goes down again and you can go up if you want to keys dot arrow dot perform and you'll go up in the search and then if I wanted to get out of the search you could send the escape key so we'll send keys via the action chains the keys we're sending is keys dot escape and then perform that and then it'll take us back out of the page now we're back into this normal Selenium web page okay so the other option instead of using release just to be aware of it's usually easier to just use release down here when you have the key down just do release so that you release all the keys the other action that you can use just so that you're aware of it is key up so you can release whatever key it has down we can also do key down keys dot control here we'll do that search shortcut again then we're going to send keys with a K and then we can do key up if you want to instead of release and just specify that keys dot control key again so if you perform that you'll get the search box up again as well here in the browser so those are two options key up key down or key sorry key down key up or key down release both of those will work alright okay so in this lecture we talked about how you can use action chains and send keys and action chains to work with hot keys or shortcuts on web pages if you need to test that out thanks for joining and we'll see you in the next lecture hello everyone and welcome to our overview of our second guided project for this Selenium course in which we'll be building a form filler the goal of our second project will be to fill out and submit the booking request form at www.thegoldbugs.com this is our test website that's basically a fake banned website the booking form has text fields it has check boxes and it has drop downs so it has a whole bunch of stuff that we've practiced in the previous section with Selenium the booking form looks like this so like you can see pretty standard and we'll go into more depth into it when we open it up in the guided code along section in this project what we're going to use is we're going to use what we've learned about identifying elements in the HTML source code or the DOM to select the form fields of interest to us we're also going to use what we've learned about filling in forms to interact with the various elements so clicking on things, selecting items from a drop down menu checking check boxes or radio buttons so on and so forth finally we'll submit the form because that's of course the purpose usually of filling out any given form is that you want to submit it and then move on to some other action everything we do in this project applies to other forms that you might see in real life or on other websites so booking forms for hotels or airline tickets, surveys searches checkout pages where you're filling in credit card information or something like that so all of that can be applied to other websites now that we've talked about what this guided project is let's dive into the code alright so you should have your jupiter notebook open to start off with and then obviously the first thing we want to start with in this project is we need to import everything that we need so from selenium let's import the web driver it's our basic object that we need or module that we need then from selenium.webdriver .chrome.service we need our service class so import service from selenium.webdriver .common.by we import by so that we can find elements on the page and then finally we're going to be interacting with a drop down element so we want that select class as well so from selenium.webdriver.support .ui we'll import select okay so that's everything we need from selenium from our settings file let's import the chrome driver path as always and then we're going to be doing the form on the goldbugs website so we need the goldbugs url as well so run that cell and then we've got all of our imports there as always we need to set up the service so the service is service executable path is that chrome driver path driver is equal to webdriver.chrome service is equal to service run that let's have that driver start up there and then let's we can do driver.maximize window although actually let's tab this over to the side so that's easier to work with and have the code side by side and then we can get the website that we're going to take a look at so goldbugs url so let's do that it opens it up over here and let's open up our chrome dev tools as well while we're waiting for it to load okay so here we go and then let's zoom in so that this code is easier to see okay great so we've got our chrome window open we've got the goldbugs website up and as mentioned we're going to fill out this booking form so if you scroll down you'll see the booking form down here do you want to book the goldbugs for your own show private or public and we've got all of these fields so we've got some text fields message, checkboxes radio buttons drop down drop down menu here and then some radio buttons and then a submit button as well so let's first narrow our search if we let's click on this so click on this let's find this div here and I think this is in a form let me just take a look at this real quick inside the div there should be a form element that's what we want to look for div form wrapper form inter wrapper and then we've got this form, okay great so we've got that here so that's the first element that we're going to drill down into so that we're not messing with other things on the page and there's only one form on the page so we don't have to worry about selecting other things form element is going to be driver.findElement we're going to search by tag name and the tag is form so that'll just get the first form element on the page like I said there's only one so that's the one that we want to be working with and it's just so that we don't have to worry about maybe some other messing around with other things on the page so now we've got this first name element so let's click on that this is input class field element and it's got a name of fname so that's how we're going to search for it so fname element is equal to driver.findElement by we're going to search by name it's called fname so again that's the attribute the name attribute that it has so let's fill this out so we're going to send keys to that so fnameElement.sendKeys just make this john ok so now as you can see we've filled out the name and the first name let's keep going fill out the last name so that's let's see this has a name of lname last name another good way to find that so we'll search by name again driver.findElement by name lname and then let's send some keys to fill out the last name so sendKeys doh again you see last name has been filled out and we'll just keep going one by one through all of these form elements click on email input ID it's kind of got a lengthy ID but it's got a good name so name email that's good we'll do it by name again emailElement driver.findElement by name email and then we'll send keys to fill out the fill out the email field so john.doh at mail.com subject and this one does not seem to have a good name so it doesn't have a name but it does have an ID so we can search for this by ID so subject element driver.findElement by we're going to search by ID not by name and then we can just paste the paste the ID in there so again you can just click on this form dev tools copy it paste it into your jupiter notebook send a subject to the subject element so send keys again our subject is what are we going to call this love your band exclamation point great and now the message if I click on this this is not going to be an input class field element ID text area and we'll just search by tag name change things up because there's only one text area element on this page and inside of our form as well so let's see message element tools and actually I just realized I've been using the driver to find these things but you can search inside the form element um like I mentioned before you can find elements on web elements so we'll search by tag name inside the form and again that's to make sure we're not searching for text areas outside the um outside the form so we've got that text area message element let's send some keys to it so we'll say you are the greatest double quotes because we got a single quote here with your you're the greatest I love your band exclamation point send that keys shows up in the message okay so we've done this first part of the form here filled out all the text boxes and the text area here which is a new element but like you can see it basically works like any other input element or text box element so that's all fine now the next thing we want to do is click on these check boxes so what we're going to do let's say we don't want a response we're just saying that we we really love their band this does not have an ID but it does have a name so let's search for this element this check box element by its name so response element we could say response check box is we're going to search inside the form element then let's find the element by name and the name is what we copied from the chrome dev tools right so it's a check box with the long string of characters after it and then we want to click that so response element dot click alright so now we can say response element dot is is selected is that true that's true right and we can see that here it's checked so our check box is good to go now let's move on to the radio button let's click on this and we want to do we want the band to attend a public show so it's input type radio and it's got this long name as well so we can we can do that so name so we're going to say what is this radio element it's going to be equal to form element dot find element by name we're going to search by name again in case the the name that we pulled from the DOM there so that's good to go shift enter to run that got the radio element and then let's click it so that we activated click run that let's see okay so I took a look at this a little bit closer and I forgot about this both of these radio buttons actually have the same name so if you look at this the private show the name is radio 560 0 f9 so on and so forth and then if you look at the other option it's got the same name as well radio 560 0f9 so in fact name is not a good one to use to select that public show radio button what I think we'll do instead is let's get the copy the X path and we'll use that instead to find this to find this input element or find that radio button element so let's say button X path SQL we can copy that copy that X path and actually I don't like this X path I think the ID on this changes when you click on it on the page so that's that ID isn't actually very very stable so what we'll do instead let's just say at input is equal to right now at at value we want this to be at value is equal to public show that should be good enough so we just want to find the input item where the value is public show that's what we want to click on so then we can say radio element is equal to the form element find element we're going to search by X path and then we can search by that button X path we should be able to find that we got that ran successfully let's click on that public show okay great so we change it up now we're not clicking on private show we're clicking on public show great keep going down the form the next item that we have is this drop down menu so if you remember from our lectures we can use a select element on this so let's get the select element so select element is form element dot find element we're going to search by what are we going to search by I think for this one we can search by yeah we've got an ID so let's search by ID copy and paste that string and then we can make that a select object so select is equal to select capital S select element great so now if we look at the options we've got all the how do you do this again we have to do the text so let's do option dot text for option in select dot options so we do that Google, Facebook Instagram and other are the options that we can select there so let's do select dot select by value so we'll do that by the value here and we'll do other select other there like you can see how did you hear about us other that's filled out we're good to go the last thing we have to do is select the survey radio button will you tell your friends about our band strongly agree input type radio value of two so for this let's see survey radio element is going to be the form element find element we're going to search by let's just after doing the last looking at the what was it last radio button I just want to make sure that inputs actually have different names so BC3 it looks like they have the same names so what we're going to do is do it on value and so strongly disagree what is that value negative two okay right so this is negative two negative one zero one two I want to do strongly strongly agree so we're going to do this by an X path again and the X path is going to be what is this this is an input element and it's got a value so at value is equal to two so two is going to be strongly agree that run successfully so we've got that survey radio element and then we can click it right that's all we have to do with the button okay like you can see strongly agree is selected now all that's left to do is submit the form so you can find this button it's not hard to do right you can use your chrome elements input class button value submit type submit you could search for things that way but Selenium actually gives us a nice method if you have a form element selected so the tag name is form element it's a form right it's the html tag but we can actually just do a submit method on it and then that will submit the form for us that will run the submit action on the form and now you can see you should have seen on the page it said submitting we scroll up do you want to book the gold bugs blah blah blah thank you the form has been submitted right so let's talk about everything that we did here we really did a lot of stuff there were a number of text fields that we had to find and fill out and those basically just sending keys to them and filling out a text area which is a new tag that I don't think we talked about or worked with before but pretty straightforward again just sending keys to it then we kept on going through the checkboxes the radio buttons we used xpaths we used names we used IDs we used the select element as well the select object so that we could just select things by their value, by their index, by their the other one visible text I think is the other one and then we learned about this new submit method on form elements in Selenium that's really useful when you don't want to find a button and you just want to submit a form very easily so that's the end of our guided project I hope you appreciate that we really went through a lot of stuff here and that you've learned a lot about Selenium in the past few sections so congratulations on finishing this project and we'll see you in the next lecture hello everyone and welcome to our section on weights in Selenium we're going to start by talking about implicit weights most modern web pages will have elements on them that can load at different times if you think about running a search for example you may have the search bar with your search text in it but you may be waiting on search results to show up on the page this can cause problems for us in Selenium if we try to find elements that don't yet exist on the page but they might in a second or two Selenium gives us a way to address this problem with what are called weights weights can be either explicit or implicit we'll talk about explicit weights in the next lecture but for now we'll focus on the implicit weights implicit weights are basically a session setting during the life of your web driver object or your Selenium session what an implicit weight does is it tells Selenium to continually check the DOM for some set amount of time for any element that's not found immediately so when you use driver.findElement for example if the element's not there Selenium will basically pull the DOM or the web page for some amount of time until either the element is found or the request times out based on the implicit weight that you set generally speaking it's a lot better to use explicit weights than implicit weights or using the time.sleep which is something of what you might call a beginner method to wait for things to show up on the web page with an explicit weight compared to an implicit weight you'll only be waiting as long as you have to and your program will run faster than if you use time.sleep for example additionally implicit weights in your program don't provide information don't provide the reader any information on why or when we're waiting for objects to show up on the web page let's say for example that we're filling out a form to check out and we're inputting our information and we have to wait until the submit button is clickable once we filled out the information with an implicit weight we're only waiting for that to happen but we don't know that's what we're waiting to happen with an explicit weight we can specify that we're waiting for that button to be clickable you can set the implicit weight in Selenium using the implicitly weight method and then passing a seconds parameter in the code what this looks like is you have all your imports and setup and then when you instantiate your driver after you do that you can just call driver.implicitly weight and specify some number of seconds such as 10 seconds let's head over to our Jupyter Notebooks to see how this works in practice alright so here we are in our Jupyter Notebook we've got all of our imports and by and we're going to be taking a look at the books to scrape URL so you can import that from the settings file set up the service, set up the driver, get the web page and now that your driver is active we can set the implicit weight by just calling driver.implicitly weight and then passing some amount of time so here we'll say 10 seconds and now this setting has been set for the entire setting or for the entire session rather so anytime that we try to find something on the page if it takes a while to load then we can Selenium will wait for that set amount of seconds so 10 seconds here until the item, the element either loads or fails to load and then we'll get an error so let's say let's find an image you know let's just image element since this is the books to scrape website and has these images down here so we'll just say driver.findElement let's look by tag name we'll just get an image tag so that just gives us one element right now if this page we're slow loading or something like that as I mentioned the implicitly weight by setting that Selenium our web session would wait until the item loaded or failed to load and then thrown error so there's not much more to implicit weights than that it's something that if you are going to set it you want to set it at the beginning of your session so that applies to everything you're looking for but as we'll see in the next lecture it's generally better to use explicit weights instead of implicit weights in Selenium just because you can be more specific about what you're waiting for so thanks for joining this lecture and we'll see you in the next one where we talk about explicit weights in Selenium. Hello everyone and welcome to our lecture on explicit weights in Selenium as mentioned in our previous lecture generally speaking explicit weights are much more useful and better to use than implicit weights where you're specifying some general time threshold for waiting for objects to load on a page. Instead explicit weights specify certain conditions that need to be met before proceeding in your code for example if you're trying to fill in a form it might be smart to wait for the form to load on the page before you start filling it in. Setting an explicit weight in Selenium generally involves two different objects a web driver weight object and then an expected condition Selenium provides a number of methods for these expected conditions on the page. You might want to use title is if you're waiting for the page title to be a specific string you could be using visibility of element located waiting for a certain element to be visible on the page or you could be using something like element to be clickable so let's say you're filling out a form and you have to fill in the required fields or click a box with the privacy policy and terms and conditions before you can submit the form in that case you might use an element to be clickable expected condition there are a lot more and we'll see some of them as we go through this section but as always refer to the Selenium documentation for Python if you want a full list of all expected conditions that you can use for explicit weights explicit weights in general are characteristic of better and more maintainable Selenium code because they are much more robust against network conditions for example so slow loading web pages then the typical alternative or what you might call the novice or beginner alternative which is to use time dot sleep before you try to find a certain element you may also want to put your explicit weights inside of try catch and finally blocks in case you want to spit out certain exceptions if something breaks down on your page let's say you can't find an element and you'd like to alert the user that the element couldn't be found because maybe that means the web page has changed and you need to change your code in terms of some new imports that we need to use this functionality we'll have to import the web driver weight class from selenium dot web driver dot support dot ui and then from the support module we can import the expected conditions module as ec that's typically how you see it shortened in python code to explicitly wait for an element there's a few things going on here and we'll compare this to our code without the weight so if you see below here the way we used to do it is to just have driver dot find element to be by id and then pass the string id but if you're waiting for the element to load in your python code this line of code might break to do this with an explicit weight what you can do is say web driver weight pass the driver as your first parameter and then some time and seconds as your second parameter and that'll be how long you wait for the element to load or be clickable or be visible after that you can write dot until and this is where you pass your expected condition so with ec which is the expected condition dot element to be clickable so we're waiting for an element to be clickable and then you pass the tuple of the identifier so maybe you're searching by id or by name here we're searching by id and that'll be the element that the web driver waits for to show up let's head over to our jupiter notebook to see how this works in practice so for this lecture we're going to do things a little bit differently because it's easier to show how explicit weights work and are beneficial to us if we just use it with a plain vanilla python script if you run it in your jupiter notebook the jupiter notebook kind of interferes with the way the explicit weights work you don't see the benefit as much so what we're going to do is we're going to write just a plain python file and then run that and see some of the errors that we get and how we can fix that so in terms of setting things up just have all your main imports so from selenium import web driver import the service import the by class then from our settings file we can import the chrome driver path and we're going to take a look at the selenium docs search url so if you go to that declaration in our settings file and we can actually take a look at this so I have I have this up in chrome right now and it's basically just this page it's the selenium python docs so we can type something into here and search for it so the first thing I'm going to do is let's get this input so the input the name is q so we can type that out in our in our python script we're going to do this a little bit in parallel right now so the input element we can say that's driver.findElement and we're going to look by name and then have this be q so if I go back to selenium in chrome you'll see here in the html the input type is text name is q so that's the one that we're looking for right now and then what we're going to do with this is we're going to search for something so let's search for explicit weight because that's what we're taking a look at in this lecture when I hit search it searches for something right it searches for what we typed in the search box and if you notice it took a while to load on this page right it took a second or two to load and then we've got this list here with references to our explicit weight in the documentation but the search results came up and the rest of the page stayed the same so if you look at this div down here and let's maybe go in one more not up here down here in the docs or in the DOM you've got this div id search results so if I go back there we still have this div the id of search results but there's nothing in it until we hit search and then something loads into that div with the id of search results and it's all this stuff here the explicit weight so this is a good example page of where we might want to use an explicit weight if we're waiting for the search results to come up so what we're going to look for let's just look for this first link and we can let's do a right click and let's copy the xpath and that'll be our xpath string that we use in our script here so what we want to do is let's do input element dot we're going to send keys and what we're going to send is what we just looked for so explicit explicit weight is what we're looking for and then this is a form element so like we learned in our second guided project we can just submit this use the submit method on this now what we're looking for in that result so the result element is we need to look on the page driver dot find element and we're going to search by the xpath that we copied and then let's call this link xpath and that's going to be the link to this 5 weight so when you look for explicit weight on this page you expect the first link to be 5 period weight so the link xpath is we can just copy paste so control v and again all I did was I got that from the chrome dev tool so you right click here copy copy xpath so that's the result element that we're looking for we're looking by xpath and we're going to do link xpath is the string we're looking for and now what we can do let's just print result element dot text so that's what we've got up there so in an ideal world what would happen is we would run the script and then we'd output the text but because of the way that web page loading works this result element what should happen when we run this script is it will fail because it won't find this result element that's 5 weights that we're looking for and won't print the text so let's run this and see what that error looks like so run the python file it'll open things up then it'll submit that searching search results and now let's go down here you might see just for windows users you might see some of this this USB device handle win error failed to read the scripter from node connection I did look into that because I ran into that error after coding on windows when I was coding on linux before and from what I can read online it's not a serious error you can just ignore it but if I scroll up here what I'm going to see is this error message no such element exception no such element unable to locate element method xpath selector and that's that string selector that we passed in there so what this is showing you is that our script as it stands right now is running into an error trying to find that element because it hasn't loaded on the page yet so that's why we need explicit weights so what we're going to do now is we're going to we're going to rewrite this script so that it uses explicit weights so in order to do that we need a few more imports like we talked about in the lecture so from selenium dot web driver dot support we're going to import expected conditions or import that as EC what else do we need from selenium dot dot what web driver dot support dot ui import we're going to import the web driver and what else from and then we're going to import an exception to because we're going to write this in a try try accept block so from selenium dot common dot exceptions let's import the element not visible exception so there's a whole bunch of these we won't go into it if you're interested in knowing all about the selenium exceptions you can read the documentation we're just going to take a look at this one for now so now we've got this input element that we're going to find in the page and then we're going to submit this form and then we've got this result element here but like we saw the result element doesn't work and we try to find it because it's not on the page yet so what we're going to do let's have a try catch block here scroll this up a little bit make it easier to see so the result element in this case what we're going to do we're going to have that web driver wait pass the driver and then the time we're waiting for let's wait for 10 seconds until and then this is where we put in that expected condition so I'm going to say presence of let's do presence of element located so that means it's found it on the page we're looking by xpath and we're passing that link xpath string again so if I go back to my is this the one that's the automated one if I go back here this is the what we're looking for here so 5 waits so let's go back to VS code or whatever editor you're working in so that's how we do our explicit wait and then we'll do what we tried to do before so print that element result element dot text which should be 5 period waits so then what we can do so we got try so accept let's see element not visible exception and then we'll just say print element not visible so we can handle that error if it crops up and then finally after all of this I forgot I wanted to just to keep things open and have the script run a little bit slower I'm going to use time what am I importing from this import time that's what I want again you don't use this in the real world but it's just so that the script maybe runs a little bit slower so that we can see what's going on so let's put time dot sleep 3 seconds there time dot sleep 3 seconds there time dot sleep 3 seconds there before it submits and then I won't use it when we're doing the explicit wait because that's the point the explicit wait is that that's what's waiting for the element to load we don't have to call time dot sleep but then at the end just to keep the page open for a little bit I'll keep it open for 3 seconds and then we'll quit the driver since this is a script not a notebook we'll quit the driver so that we don't end up with it hanging so like here if I go down here because the script failed the driver didn't quit so close out of that so this is the way it works now we've got a nice try accept block here try accept finally so let's run this and see what happens so it opens up the chrome window it's the documentation it's going to wait then it sends the text next it submits and then it closes if you see here we don't have that error the presence of element okay there was one error I forgot you have to pass this as a tuple this is so let's add that add that as a tuple here and run this code so it opens up chrome here we go again sorry about that opens up chrome to this page then it'll send the text right then it'll submit the search results pop up and then it'll print the text so here that's what we've got five weights right that's what we were looking for if you go up here as I mentioned before there's no we don't get that element not located error you still have these two windows errors but again you can ignore that so now our code has worked perfectly well right it's handled that if I exit out of this and go back to our script here by using the explicit weight inside of a try accept path so if there is an exception we'll catch that exception by using the explicit weight we are able to capture that result element even though we're waiting it for it to load on the page so this should help you see why explicit weights are really powerful compared to implicit weights because now we know exactly where we're waiting for something to load it's not on this original page or while we're sending keys or submitting the form we're waiting for the results to show up before we're checking it and so this is a pretty pretty good manner of handling that in Selenium so again in this lecture what we took a look at is how you use explicit weights so for that you have to remember to import your expected conditions import your web driver weight if you want to catch exceptions you can import some exceptions from that module in Selenium and then you can use this pattern of web driver weight driver and time until that expected condition is satisfied and then pass a tuple with those locators so the buy method and the string of interest and that should help you help you avoid running into these timing issues where things are loading on a page thanks everybody for joining this lecture about the explicit weights in Selenium and we'll see you in the next one hello everyone and welcome to our lecture on adjusting network settings hello everyone and welcome to our lecture on adjusting network settings in Selenium sometimes while you're working in Selenium you may want to emulate slower network speeds in order to better understand the user experience of your site or maybe the user experience of some site that you are working with Selenium Chrome tab tools does provide some tooling for this so there's a network throttling setting in the network tab you can use this for example to emulate a slow 3G connection on a website however we can't access this setting in Selenium from our python code so it's not always a good thing to use if you want to emulate different network conditions the way you can automate it in Selenium is with the set network conditions method on your web driver object set network conditions has several parameters that we can fill in the first is a boolean for offline so you can set the network basically to be up or to be down I don't know why exactly you might turn it offline but that's just to be aware of that that is one of the parameters you can set the network latency in milliseconds you can upload and upload throughput in bits per second as well by adding this latency or lowering and changing download and upload throughputs for example you can emulate different types of networks and especially slow networks if you want to know what the experience is like for users on those types of networks to use this type of functionality you can have all your imports and surface and driver setup as usual and then you call this method on your web driver object so you can say driver.setnetwork conditions and then fill out the parameters so here we're keeping the network online so offline is equal to false setting some latency in milliseconds and then setting the download and upload throughputs as well and remember this is in bits per second so you're going to have to do some multiplication just to make things realistic because otherwise you might end up very very slow otherwise wouldn't want to be let's head over to our jupiter network our jupiter notebook now to see how this works alright so here we are in our jupiter notebook we've got our normal imports of web driver and service we don't have buy because we're not going to be looking for anything on the page today and then the two URLs that I imported are just the selenium URL which you can see in our chrome window there and the python URL as well because we'll be switching between these two sites set up the service, set up the driver and then get that selenium URL so to change the network speed you can just do this on the driver so all you have to do is use driver.set network conditions that's the method that you want to use and then there are a number of different options so one of them is offline or parameters rather offline we can set this to true so this is going to set our network offline right latency this is in milliseconds so that's additional latency beyond whatever your normal network conditions are and then you can put download throughput download throughput and upload throughput and this is in bits per second so if I wanted to do let's say 500 megabits per second 100 times 1024 and we can do that for download as well so we run that now we've changed the network conditions so if I try to get a get the python URL go to a different URL I'm going to get this error that says there's no internet right and this will throw an error in here as well so message blah blah blah error internet disconnected so that's because we set offline to true so our network is offline now so in order to fix that we're going to have to turn it back on so we can do driver.set network conditions then let's just do all of this again so offline is equal to false so it's back online latency we can add maybe let's add 5000 milliseconds so 5 seconds of latency download throughput we're going to put this let's cut this in half so 250 times 1024 and then upload throughput we'll just say keep it the same since we're not uploading we're not uploading anything so set that as the network conditions and then let's try driver.get python URL and what you'll see is that it takes a lot longer for this page to load so if you've got a normal stable internet connection it should load pretty fast and we'll try it again with another website when we reset the network conditions but you should see that it took a really long time and that's because of that latency that we changed and the download throughput that we changed you can also get current network conditions if you want to see what they are or what they're set to in Selenium using get network conditions and then that'll output a dictionary with the speeds or with the value of the values of those those parameters so now if we change the conditions again so driver.set network conditions and do offline is still false and latency, remove the latency and then bump up the download and upload throughput so we'll make this this is 500 times 1024 and just do the same thing for the upload throughput too so you know reasonable pretty reasonable web speeds and now if we just get the we go back to that Selenium website you'll see it loads a lot faster than the Python one that we went to and that's because we turned off the latency so okay in this lecture we went through how you can change and change your network conditions if you want to test web pages under slower or faster conditions, change things up and how you can see what your network conditions are so the two methods of interest to us are the get network conditions and then the set network conditions thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on setting up mobile views in Selenium we can automate the emulation of certain mobile devices using our standard Selenium web driver if we want to do this in Chrome all we have to do is add mobile emulation option to our Chrome options object and then specify a type of device so maybe it's an iPhone 12 or an iPhone 14 Samsung Galaxy whatever the options are for your particular version of Chrome or your particular web browser if you're using Firefox or Opera or one of the other ones just a note on this section since it will be a short section the information of mobile testing is not always straightforward in Selenium so you may need to use a mobile specific package to achieve what you're trying to achieve for example I know there is a Python package called Appium that's basically Selenium but specific for mobile testing of apps and things of that nature what we're going to find is that Selenium may not be the best choice for mobile testing but there are some options that you have if you need to do something basic on a website to set up your Selenium session to emulate a mobile device what you can do is import and start your driver and service as usual and then you'll have that Chrome options object that you instantiate and you're going to use the method which is add experimental option and the parameters here are going to be mobile emulation so that's the option that you want to activate and then you'll pass a dictionary the key in the dictionary will be device name and then you can pass a string specifying whatever device you want to emulate on your particular version of Chrome let's head over to our Jupyter notebooks to see how this works in practice alright we're starting off in just a normal Chrome window with our dev tools open for now because I want to show you what we're actually going to be doing with these Chrome options that we're going to use with the mobile emulation basically what we're doing if you look at this Chrome toolbar you've got the normal elements page that we're used to seeing we've got the select an element in the page to inspect it so we can do that click on it and I'll show it in the DOM the other option we have is the device toolbar however this will give us is various ways of and let me exit out of this for now actually let's go back so the dimensions that you have here you'll have a whole bunch of device options that you can emulate so we can do an iPad Air and that'll give you the dimensions of the iPad Air iPhone SE, dimensions of the iPhone SE let's see Samsung Galaxy that's got the dimensions there so your web page might be laid out a little bit differently than it usually is but that's how you get that mobile view just using the Chrome dev tools so that's basically what we're going to be using in Selenium all we're going to do is pop it open using that automated Selenium those automated Selenium options so here we've got our Jupyter Notebook open we've got Selenium WebDriver service imported from our settings we're going to import the Chrome driver path and then we're just going to go to the Selenium URL to see how that looks on various devices so we can import that start the service and then actually the first thing I want to do here so I'll make another cell is you want to pass the dictionary with these options so we're going to do mobile emulation and the device name that I want to use is I think let's see we can use one of these I think iPhone 12 Pro is an option so we'll just do that let's so iPhone 12 Pro is going to be our option there so do that and then we can do the Chrome options so pass that to our Chrome options object we need WebDriver .chrome options instantiate that and then we can say chrome options .add experimental option the option is mobile emulation the parameters for that come from our dictionary so we've got that that all works great so now what you need to add is that options parameter like we've seen before with some of the other chrome options that we've used get rid of that space start it up and it pops open this chrome browser and then if we get the Selenium URL pop this over you'll see what we've got here is basically a mobile view of it so this kind of looks like an iPhone so you're still running Selenium as normal in this browser window but the window size has now made the website smaller so that it's basically the mobile view and specifically the iPhone 12 Pro view of that web page and so then of course you can let's quit this you can do a different mobile emulation option so you can do mobile emulation do this again so device name is going to be you making an iPad mini for example like I mentioned it depends on your version of chrome so you have to check that out to see what options you have so chrome options is equal to webdriver.chromeoptions and then chromeoptions.add experimental option and we're going to do that mobile emulation with the mobile emulation dictionary there and then we can pass that start up the start up the web driver again with those new options and then you'll see a new window with a new window size chromeoptions run that it will open up chrome and then we can do driver.kit seleniumurl and now this is like an iPad mini so you can see it's not the normal selenium website you've got this mobile view drop down menu there and all this other stuff so that's just a basic overview in this lecture of how you start off with mobile emulation if you need to do that the gist of it is you just have to add this experimental option of mobile emulation and then pass the dictionary with a device name as the key and then some sort of value that comes from your version of chrome thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our lecture on mobile actions with selenium as mentioned before in the previous lecture selenium is not always the best choice for mobile testing although it is possible to use it for that purpose so as we saw before you can pass certain chrome options to emulate mobile devices for example and certain web page actions something like clicking on an element or clicking on a button really work just like with anything else it doesn't matter that you're in a mobile view of the page other actions however might require that you use the execute script method with javascript and that's why other packages might be preferable depending on what you're using it for if you don't know javascript for example and like I said before this is not a javascript class then you might want to look into other python packages that could better suit your needs rather than trying to learn javascript to code around the limitations of selenium and when I talk about those limitations of selenium it's things like there aren't really actions that you can use for mobile specific actions things like using two fingers to click on something or hold your screen or three fingers on your screen things like that there isn't really that capability in selenium that said we'll head over to our jupiter notebook to get some practice with the execute script and some other methods just so that you can see the boundaries really of what you can do for mobile using selenium alright so we've got our jupiter notebook open and we've started it with some of the options that we used in the previous lecture in terms of imports to start off with you need web driver and service you should also import by we'll need that in this lecture and we're going to take a quick look at the action chains object as well the url we're using is a selenium url so selenium.dev and then we can start the service add the chrome options that we talked about last time so we're emulating an iphone 12 pro here we added that option to using using add experimental option to our web driver and then we started our driver here with those chrome options and we got the selenium url so that's how we get here to start with now what you'll see as we saw in our last lecture is that we've got this selenium view up in mobile right now right it's not totally mobile because there's no cutoff you can just kind of scroll down but it's good enough for us right now so what I want to show you is that certain actions are just basically web actions right there's nothing specific to mobile about them often times in mobile websites you might get this kind of navbar toggler icon that we've got here the hamburger menu as it's sometimes called and so maybe we want to click on that and all we have to do is that just works the same way as anything with a normal web page right so we can call this just call this a button element and we can just use driver.find element we're going to look by the css selector and the string we're looking for is a button .navbar toggler so that's the class on this button here it's at the top of the page so it should be the first one so run that we've got our element and then what we can do is click on it so you clicked on it you should see here that menu drops down right you can click on things in the menu and it's still that mobile view of a hamburger drop down menu and the way these usually work is you click on it again and then it will close things back up right now you're back on the normal page so in terms of selecting and clicking on stuff in mobile sometimes it's just basically the same as with with a normal web page it's just that you've set up your chrome session to use those emulation settings as chrome options so in terms of touch type actions like I was talking about maybe to scroll or kind of flick with your finger right double tap something like that where a mobile device could have some sort of specific action that that interaction generates right like a screenshot or something I don't know we can try it with action chains but what we'll see is we'll just get some errors when we when you maybe try to do things that you would otherwise think would work so we can set up an action chains object and what I'm going to try to do is scroll to this this getting started header here in the middle of the page so that's just a basic scroll action right it's nothing too complicated so let's see how that works using using the action chains so first I'm going to specify that element it's an h2 element so let's get that let's see h2 let's get all the h2s on the page driver dot since it doesn't have a there's no id or name on that h2 so driver dot find elements plural we're going to search by tag name and we're looking for that h2 tag there should be multiple on the page but we're just going to get the first one getting started what do we call this getting started element and this is h2 this is from the h2 elements and it's the first one so zero index it's just that now the action chains object has a scroll to elements on it so on this page you might think that if we scroll up to the top here stay started up there that we might just be able to use that and specify the getting started element and then dot what is this dot perform to perform the action but we get this kind of error here which what's it say invalid argument invalid parameters let's take a look at yeah invalid parameters so you can try to perform it there is only that one parameter that you need and yet you still can't do it using an action chain in selenium so instead what you have to end up doing sometimes is using this driver execute script so we're going to execute some javascript so you can use windows scroll to and then 0100 so that's just like an x and y value here window to scroll to is just a javascript scrolling function so we can do that and then it kind of scrolls down right we can maybe do it again execute script window dot scroll to 0 like 800 right do that and scrolls a little bit further down to more or less where the getting started header is but as you can see we've kind of got some limitations really to using selenium for emulating mobile actions or emulating mobile devices as well like I said before this isn't a javascript class we don't want you to have to write a whole bunch of javascript inside of your execute script because it makes your code hard to debug and it means you might have to learn another language too instead of just using selenium so if you are using selenium for mobile testing or you'd like to try to use it we just want you to be aware of some of these limitations of using it and you might want to check out some other packages like apium if you're going to get deeper into trying to use python for mobile testing unfortunately selenium doesn't have a whole bunch of great functionality for us here ok so hopefully that was an interesting lecture for you guys again it's a little bit of knowing the boundaries of selenium rather than getting deep into functionality but should still be useful if you find yourself needing it you'll know what not to use it for as well as what you can use it for thanks everybody for joining and we'll see you in the next lecture hello everyone and welcome to our section on the page object model to start we're going to talk about how you might do page testing in selenium without using the page object model before we take a look at what the page object model is the design pattern and how to implement it in python we're going to try to test a page using what we've learned so far specifically in this lecture we're going to write some code testing the search on wikipedia's main page so before we get into the page object model we might want to think about what are some of the drawbacks of not using a design pattern for testing our web pages some of the potential drawbacks that we're going to see as we go through this testing exercise of scripting things so to speak or writing things in a jupiter notebook versus using the page object model are as follows the first is there's this splatter you might say of locator types and strings across your code so the locator types and the strings for finding page elements are just going to be wherever you need them they're not going to be centralized in one place and so if the page changes or the id of an element changes for example you'll have to search through your code to figure out where exactly it is that you're using that locator type or that locator string the second thing that you'll find is a drawback is that you have repeated code in a lot of places so for example let's look at finding an element and sending keys for filling in a text box that's really two steps in Selenium so it's finding the element using some sort of locator type and string and then you have to send the keys now if you have a large form to fill out or a large form to test filling out you'll have to repeat both of those steps for every input element that you work with obviously that's not ideal right and with the page object model as you'll see later we can actually avoid that and cut down quite a bit of repeated code and finally the last thing is that it's very difficult to scale your Selenium testing code if you're not using a page object model so you'll just have all of these scripts that maybe work maybe they don't as the web page changes and it'll become very hard to maintain and that makes your testing code difficult to scale so with that said let's head over to Jupyter Notebook now and what we're going to do is we're going to take a look at the Wikipedia search and test it just using just coding as we've done before mostly in our Jupyter Notebook alright so we've got our Jupyter Notebook open and we've got the web driver open as well to the main page of Wikipedia so the imports we just need web driver service and buy and then from our settings file we're going to import that Chrome driver path the Wikipedia URL which you've got here that's just that main page and then we're going to use another search URL in Wikipedia later on so you can import that as well that's the Wikipedia search URL set up your service, set up your driver get the URL and now we can get going so let's say that we're trying to test this website and what we want to do is to check to make sure the search functionality works right so we're going to search for something submit it and then check that the result is what we expect it to be so we're looking at this so we want to get this input, this search element here first so the search element and just keyboard is off so search here we go search element is going to be equal to driver.find element and we're going to search by name if you look at the HTML there it's got a name of search so that's the string of actually let's search by ID ID is always a little bit more reliable it's a search element so we've got that now to do a search we have to send a key we have to do send keys to it and I want to look for let's look for python and then that's in there and now let's check this search and this will give us some results back it's an input element so we can just run the submit method on it and it should run that search for us and now we've got this page here with some results on it we were looking for something more specific but now we've got this page here that we can take a look at and so we want to make sure that on this page maybe for our test we want to make sure that there's a link for the python programming language since we're using python we're going to check for that so what we're going to do let's copy the xpath here and we want to make sure that that element exists python link element it's going to be a driver.find element we're going to look by xpath and the xpath is actually I don't like that because it's a little bit too specific but it's a link so what we're going to do let's do double backslash we're going to look for a link on the page where the title is python programming language which is this attribute here it's a title python programming language I think that's all we need so let's run that okay so we got the link and now if we wanted to have kind of a test a test like thing or a test like cell in here then we could assert something so we want that link element to exist and if it doesn't we'll throw an error so link for python not found so there's nothing there because the python link does exist if it didn't we can just do this really quick make this a none instead and then run this cell again and you get an assertion error because the link element is now empty there's nothing there obviously there is something there but we made it none just to to show that that assertion is working so now we tested the main search but maybe we want to test the search on another page so we can get that Wikipedia search URL so we tested one search let's test this advanced it's more advanced search here so we have to click on something to open up the advanced search so we want to make sure that works maybe and let's see let's click on this is this a label yeah this is a label so we can we want to click on this label so let's copy that x path so we'll call this click x path and that looks right to me click on it again to close it up so we'll call this test element so that we can click on it so find element we're looking by x path and the string is that click x path and then what we want to do we want to click on it like I said that's how you open and close it okay so we've opened up this search form and now we want to we want to pass some search text so we're going to look at this input it's got an id there so we can just copy paste that so these words we'll call it these words element because they're searching for these words driver dot find element we're going to search by id got that id there we copied so we got it and then we want to search for our python so we're going to do the same search but from a different page and actually to activate this I forgot you have to end it with a comma I think and then you get that bubble kind of appearing there that's the word you're searching for so now we've got this search form but we have to submit it again and now we have to search for that search for that maybe we want to search for this link on this page right so this page setup is slightly different than when we did the normal search before so now we have to get this link again let's take a look at this probably just a with a title so they I think our x path can be the same thing so now we still want to check that there's a python link element in here so driver dot find element we're going to search by x path we're just going to reuse the x path that we used before so we're searching relatively for a link with the attribute of title equal to python programming language run that so we found it so now if we assert the python link element exists and otherwise the link doesn't exist then nothing happens because the link does exist but this should kind of start to show you how if you have a lot of pages that you have to test with selenium this can start to get quite tedious and as you should already kind of notice it's not very clean so first off we've got this in a jupiter notebook which is not the best way of testing websites right it's what we've been using to interact with web pages so that we can go through step by step but in terms of scalability this is really not scalable so that's one part of it the second part of it is you've got this duplication of work really so we're sending keys to a lot of things so first we find the search element then we send keys to it then we go down here find this element send the keys to it so that's duplicated code that we can cut down on so that's another thing and then you've got these strings that are maybe string constants here like this click xpath but then you combine it here with the locator type in order to find the element maybe you don't have it so down here it's just in the find element you've got these strings that were the place so it's really not a clean way of identifying the web elements that are important to you so with all these things with all these issues that we've cropped into just trying to test some basic search functionality on Wikipedia you should be able to see why we're going to start heading into the page object model to use that to guide our website testing in Selenium alright with that said in our next lecture we're going to go through the page object model and what it is so thanks for joining this one and we'll see you in the next one hello everyone and welcome to our lecture on the overview of the page object model in Selenium as you may have seen in our previous lecture as we went through that testing of Wikipedia's search you should have seen some of the drawbacks to using a non-page object model approach to testing with Selenium or a non-design pattern approach to testing with Selenium so what exactly is the page object model well all the page object model really is is a Selenium design pattern or a Python design pattern for Selenium for building and automating your tests of webpages the page object model or POM for short typically consists of these three components there's usually a test case or test cases if you're testing multiple things on a web page there's the page objects themselves and the locator class or locator classes we'll talk about each one of these as we go forward in this lecture one visual way of thinking about the way this is set up in python is as we laid out there's three parts to this so there's the tests the page objects and then the locator class or locator classes so you start off with your test case there's something that you want to test on a web page and then the page objects feed into that test case as a way of abstracting the web page away and then you've got a locator class which is where you hold all of those tuples of locators we'll talk about these in more details with more detail one by one starting with the test case the test case in the page object model design pattern will be a class with methods for testing that a web page works as expected for example you might want to test does our search form work if we type something into it and hit search does it actually return results the way we expect it to another example might be does our checkout process stop people from submitting bad data so somebody tries to check out without using for example a credit card number or a valid credit card number will our checkout process stop them from doing that in this way the test cases are usually verb driven that sometimes what you see when you get into testing and there's going to be an assertion for the expected behavior so going back to these examples does our search form work the we're going to maybe assert that the search results we see have some sort of basic expected behavior right if we search for python for example we get a web page that says python in the checkout process example the assertion might be if you enter a bad credit card we want the page to not continue with that submission and we want to see maybe an error or a warning that you have to fix your credit card information in terms of the test case you can either use pythons built in unit test module or the pie test module for this course we're only going to use unit test and pie test won't be covered but in theory both of them will work for this design pattern now we get to the real the bulk of the page object model which is the page objects themselves the page objects contain methods related to our interactions with the web page typically what we'll do is we build a base page class to hold common page actions so think of finding or entering text on a page that's something that you could do across multiple pages on your website then after you have your base page class set up our child page classes will inherit from that base page class and combine some of these actions for page specific workflows going back to the base page class it might have a method for entering text for example in a text box but then in our child page class for our web page of interest for example the Wikipedia web page there will be a method for entering text in a specific part of that Wikipedia page but we'll get that action, we'll get that method really from inheriting from our base page class finally the locator class or locator classes will hold the tuples for the elements of interest on our web page like you saw with our non-page object model testing of Wikipedia it's a little bit cumbersome to have these locators sprinkled all over the place the tuples will usually have a locator type and locator string so that's what we've been passing to our find element method for selenium so you'll just store things really like by.id and then the id or the name that you're using to search by locators are usually put in their own classes for each web page that you're dealing with but not always so sometimes you'll see them maybe put in the page object class itself the way we're going to be doing things is that the locators will be in their own class but that's just so that you're aware you might not always see it laid out that way so now that we've talked about the way these things fit together let's walk through this diagram again so we'll start with the locator class so the locator class is basically just a python file with a class specific to a web page that contains all the locators for that web page so you'll really just see a whole bunch of tuples for the locator type and then the locator string once you set that up you can set up your base page page object class this will be a class that contains common methods so that we don't have to duplicate so much code right in order to click on something you have to find an element and then click on it so we can use that as one method for example to enter text into a text box you have to find an element and then enter the text in the text box or use the send keys method by putting that in its own class we can then use it with inheritance for other web pages and that's what we'll have here so the base page class will be inherited by the other web page so now with the locators for that web page and the base page actions we can click on things on that web page enter text in forms on that web page so on and so forth finally that web page class will be imported into our test case so that we can do whatever it is that we're trying to test so if we need to have those web page actions to test filling out a form correctly or incorrectly we can import that into our test case and then use those methods to build out that test case with all that said in our next lecture we're going to go through how to build that out in a specific example for testing the search functionality on Wikipedia hello everyone and welcome to the first part of building out our page object model for Wikipedia to build up our understanding of how to use the page object model design pattern for testing we're going to test the search functionality of Wikipedia to do this we'll need the three parts we talked about in the last lecture we'll need the test class this will be where we write out the methods for testing the search functionality we'll need the page classes this will describe the functionality on our web pages and then we'll need the locator class or classes these are the tuples for those element locators on the web page so going back to this visual here that we saw in the previous lecture our test case here will be the Wikipedia search for the page objects class we'll have a base page class and then the main page which is that Wikipedia main page for all the functionality there and then we'll have the locator class with the locator tuples for that main page of Wikipedia to start with we're going to focus on this test class in the first part of building up our test we're going to outline our classes so we're going to outline all three of those parts of our Selenium tests we're going to be using pythons built in unit test module test classes in the unit test module can have two test level methods one of them is setup and the other one is teardown first we're going to use the setup method to start our web driver in service second we're going to use the teardown method to quit our web driver and finally we'll write a broad brushstrokes methods for testing Wikipedia search again this will be a kind of verb driven method that we'll expand upon later just a note this is not a course on testing specifically or unit test in general it is really a course focused on Selenium we're only going to be using some basic functionality from unit test we really don't need to be a testing expert or understand everything that you can use the unit test module for however as mentioned in the course overview we do expect a basic python knowledge which does include some understanding of how tests work either in theory or in practice again we'll see a lot of unit test functionality as we go through this section and build out our Selenium test of Wikipedia let's head over to VS Code now and get started on building out our test alright so unlike most of our previous lectures we're going to be doing most of this coding in in VS Code or if you have another editor that's fine too you know any of them work I'm going to be doing this just in this sandbox folder that I have for writing code that really doesn't matter or that I'm not checking into the repo and then I'm going to have this POM folder that's just the page object model for this section if you want to find the code as we go through it lecture by lecture you can go to the GitHub repo and look at section 10 on page objects and so each section here lines up with the lectures and you can get the finished code from there but we're going to be starting in this POM folder so the first thing I'm going to do is we're going to make some new files so we need a locators file so locators.py we need another file which is going to be what is this this is going to be test search .py so this will be our unit test and then the other one that we need is we'll call it page.py so page.py is going to be our class for the page objects themselves ok so let me just take a look so the first thing we're going to do here let's just start off with our locators we'll call this class main page locators this is going to inherit from python's object and we'll just pass for now so in this lecture we're not going to go through and have all the locators set up there so we'll just make that a class but have it not have anything in it we can also do this for the page the page objects here so like I talked about in the lecture we're going to have this base page class inherits from object going to pass for now and then the other class that we're going to have is the main page for wikipedia's main page this is going to inherit from that base page class then we can pass because we'll fill it out in a later lecture and then we go up to our up to our test class so we're going to call this let's call this class wikipedia main page search and this is actually going to inherit from unit test dot test case so in order for that to work we're going to have to import unit test then that should be good to go pass for now we're going to come back and fill that out and then to run this to run this a testing script if name is equal to main then we're going to the way you run this unit test is you call unit test dot main and then that'll run the test on this in this in this file so we've got things kind of set up so we've got our locators we'll deal with those later and put those in those are just tuples we've got our page objects we'll fill that out in the next lecture I think and now we've got this test so like I talked about in the lecture there's a couple of important methods that we're going to set up here so two of them are set up and tear down and that's just what happens before and after the test is run and then let's do a test search alright so those are the three methods we're going to be working on to set things up what we're going to do is set up our service and our driver so for that we're going to have to do those imports so from selenium let's import the web driver and then from selenium dot web driver dot chrome dot service import that service class and then we're also going to need what do I need from our settings file we'll import the chrome driver path let me actually close this file give us some more space the chrome driver path and we're testing wikipedia so let's import that wikipedia url and then we can just set this up like with anything else so set up that service executable path is going to be that chrome driver path the driver is the web driver dot chrome service is just self.service and then the end of the setup is just getting that web page so now that the driver has been set up we can get that wikipedia url and then that'll be that then we're ready to test that search right so that's the setup on this test the tear down will be we've got self.driver that's been set up we can quit that right that's going to be the tear down so let's save this and this should be if we run this now let's try to run this now what will happen is we'll just open this up get the page and then close that's all that happens so ran test in one or five seconds one test in five seconds and there really isn't a test right we just got this test search that passes through so of course it's okay but this should show you at least the setup and tear down methods work now we're going to start building out this test search so what I'm going to search for is I'm going to search for python programming language like we saw in the first lecture for this section that's one of the pages on Wikipedia on python on the python programming language so that's what we're going to search for now the next thing is we're going to start using these page objects so I want to get that main page and what we're going to pass to it is self.driver so now if I want this to work I'm going to have to import from the page module so from page import main page and that will be how that works so now we don't have an error there then I'm going to want to search so main page dot search and a search for the search text that we just put there python programming language and this is part of the unit test module it has all these assertions that you can run so assert equal or whatever whatever we're going to use assert equal and then this is going to be search text plus hyphen Wikipedia and then we're going to get the title from the we're going to get the page title from our web page and the reason I'm going to do it this way let me go up to chrome here if I go here and I click on python programming language and you look at the title here it actually says it has that dash Wikipedia afterwards so that's how we're going to check that we're actually on the right page is we're going to check the html title so I think this is going to be it for this lecture so what we've done so far is we've set up the skeleton of our test we've got a locator class we've got these page classes that we'll fill out next and we've got the the body of our search really so this is the this is what I'm talking about about the the abstract search that's going on we've got some text that we're searching for we're going to start that main page object with the driver and then we're going to search for something and we're going to say I expect the result of this search the page as a result of this search to have a title that's going to be python programming language dash wikipedia so that's the test that we want to pass and then we've got the setup and tear down methods that will start and quit the driver ok thanks everybody for joining and we'll see you in the next lecture where we will fill out those locators and the page classes and things will start coming together hello everyone and welcome to the second part of our page object model task here where we're building out the test for wikipedia in this lecture we're going to focus on building out the page objects classes and our locator class our locator class will hold the relevant tuples for the page elements so like I mentioned before this will be a by type so by id or by css selector and then a locator string the base page class is going to hold common page actions things like getting an element on a page, entering text in an element that you found or selected clicking on an element that you found or selected for example so on and so forth the main page class will then inherit from that base page class that we've set up and it will have page specific methods for example searching on wikipedia involves two steps so we're going to have to enter text into the search form and entering text is an action we can inherit from the base page class and then we'll have to submit the form or click on a search button and again that's an action we can inherit from the base page class so why do we structure things in this way? having a base page object with common actions makes it a lot easier to build new pages and workflows from those actions let's say we want to test another part of wikipedia for example instead of having to build out something completely new we can just test the new search page for example or if we wanted to use the advanced search page on wikipedia where there's more inputs maybe more things to select a drop down menu we can just inherit that base page object again and keep building out the actions for the new pages and it will still be pretty clean it also keeps our page locators and page actions together rather than spread out in your code so like we saw when we were testing without the page object model these strings and locators can sometimes get spread out in your code as you're going if you're just using a scripting approach structuring things this way we have all our locators in one place and then those locators will be used by that main page class all of this makes your tests a lot easier to read and there's the old adage that code is read a lot more than it's written so you want to be kind to everybody else who has to read your tests including your future self it'll make it a lot easier to understand what's going on when you come back to it at a later date let's head over to VS Code and keep building out our Selenium test alright, we're back in VS Code so in the last lecture what we did was set up the skeleton for our test we've got this locators class the search test here or testing the search and then page class so we're going to start filling out things a little bit more let's start with the locators class so if I go back here what this is going to be is a class with the tuples for the important things on that main Wikipedia page so the first thing that I'm going to want to do is import the by class because I have to look for I have to look for stuff right? so we're going to import by and then the things that we want let's pull up Chrome again if we just go to the main page, click on this then what we want is we want this search input and then we're going to we also want to search for something let's see, we've got input an ID of search button we've got an input with an ID of search input so those can be the two ways we look for things so let's make these constants or uppercase so this is going to be a tuple which is by ID and the ID is search input and then we've got the search button this is also, we're also going to do this by ID and I think this is what is this search button I think let's go back to Chrome here, just take a look at it so the ID search input is that where you put the text right? and the search button here has an ID of search button so that's that that's this thing right here the magnifying glass if I go click on that that's what we're going to click on you could submit the form but I'd like to click on the button too just as as practice and then you should have two let's see two spaces there and that's it so this is our main page locators class really straightforward we just got these two constants right that are how we're going to look for that search input and that search button we don't have to do any more updates on our test search so now what we want to do is take a look at our at our page classes and the first thing we're going to do is to start building out this base page the first method we're going to have is the initialization method so that's going to hear it from self and the one thing you want to pass is your web driver just say self.driver is equal to driver again that's going to be that web driver object now we need a few other, quite a few other methods here at least for now so we're going to get a our base page is going to have a get element method let's pass and we'll go back and fill all these out in a second I just want to get these here get text then we need to enter text because we're going to be entering text in these text boxes for the search we also need to be able to click a button and we need to check the page title so get page title self so that's what we're going to fill out here and as we go through this you should start to see why doing things this way will end up helping us so the first method we're going to build out is getting an element from a page so in order to do that this is a self method we're going to need a locator type so by id by name and then a locator string that's how we're going to get the element and then the way this is going to work it's going to return self.driver it's a web driver you can do find element and you just pass that locator type and locator string and that's all there is to it that's all there is to the get element if we want to get the text what we can do see we'll call this element is self.getElement so again now we don't have to we can just use this method for everything else we don't have to say driver.findElement all over the place so what is this we're just doing the locator type and locator string so those have to be those have to be passed as well as parameters for this method and then this is getText so we're actually going to return element.text so getElement uses findElement which is why we're going to have a web element returned and so we can get the text on that to enter text let's do self.getElement we need a locator type and string locator string so we're going to do a get element based on that and then we're going to send the text we're going to send the keys so we have to have some text to enter text parameter there so that's the enter text and then let's see clickButton so we're going to click a button that's kind of very similar to what we did above so getElement getElement locator type locator string so we need those type locator string and then just click on it right and then get the page title we'll just return that's from the driver so return self.driver.title and then we've got so that's our base page object there and now what we want to do is use this base page and all of its really nice methods to build out the search on the main page here so what we're going to be doing is we're searching this is going to inherit what does this take self and then we're going to search for some specific string we're going to be searching for this this python text right that's where we're going to be looking for so add the colon there so in order to search what do we have to do you have to enter text into one of the locators and then you have to click the button to search again you could submit the form I'm just doing this way as another bit of practice for how to do this so where do you have to enter text enter text takes the locator type string and then some text and a string well we've got that already right that's in our main page locators class so we can import that so if I go up don't close that but go down here and then run an import so from locators we're going to import main page locators right that's the locators class and then what I can do here is to unpack this tuple in a second so main page locators. you want to enter text in the search input and you want to click the button that's the search button and then since this is a tuple we need to unpack it so you can do that with the asterisk in python and the only other thing that we have here is there's text that we're searching for our search string parameter here so that's where that goes we're entering the text in the search string and that is basically it I think so now let's go let's kind of work our way up we've got these locators right and they're just elements on the page these get inherited by our page module with these classes our base page class has methods that reduce the work that we have to do and the code that we have to type so if I didn't have this and I was typing this out to enter text here first I'd have to find the element and then I'd have to send the keys and to click the button first I'd have to find the element and then I'd have to click on it right by having these methods the code that you have to write you don't have to duplicate it all over the place and so if we wanted to do this on Wikipedia's advanced search page for example we could just do this again but with a different class for that page and have the same methods right enter text somewhere click on a button or submit a form and we wouldn't have to build out anything new additionally what is going on in this search how does this search work you enter text and then you click on a button that's really easy to understand it's not really confusing at all it's the way anybody does a search and then if you go up you've imported this page object to use it in our Wikipedia main page search so we've got the setup class here which is a testing method so we'll set up our service and our web driver and then get that URL to test with we've got the tear down so it quits the driver at the end and then this test search which is what we'll be testing and now that everything's been built we can run this run this code here so we can run this python file and it will test this Wikipedia search so it opens it up searches and then let's check ran one test in 7 seconds okay so it didn't fail so our test actually succeeded so we just successfully built out a python unit test with selenium to test Wikipedia search functionality so this is pretty cool and we're going to improve it just a little bit in the next lecture but hopefully you should be able to see the page object model is so useful compared to that jupiter notebook method that we were doing before alright thanks everybody for joining this lecture we'll have one more lecture focused on improving this a bit and then that'll be it for the page object model section thanks for joining and we'll see you in the next lecture hello everyone and welcome to the last part of building out our page object model test for Wikipedia in this last lecture we're going to focus on the base page class in our page objects python file specifically we're going to think about how can we improve how can we improve our git element method in our base page class one of the things that we can do as we talked about in a previous section of this course is we can use weights right now there are no explicit weights specified for getting page elements if you'd like to review pages or weights and how weights work on pages in selenium please go back to that series of lecture but what we can use is we can add an explicit weight so that if something goes wrong we'll be able to catch it or we'll be able to not throw an error in our test the other thing that we're going to add is there's no error handling so let's say we add that weight and maybe the page doesn't load in time and so we get an error on our search we'd like to print something out to let us know what error occurred and maybe some more information about it so what we can do as well as we can add a try catch or try accept block let's head over to VS code and finish up writing our selenium test alright so we're back in VS code we're just going to clean up a couple things with our test here to make it slightly more elegant slightly better looking more flexible in terms of if you have a bad connection or something doesn't load and the great thing is that we don't have to touch a whole bunch of stuff so our test here we don't have to touch right we're testing the search in the previous lecture we saw the search works the test passes so that's all well and good we don't have to touch that the locators we don't need any new locators we don't have to change this so our locators class will stay stay the same all we have to do all we're going to do is try to improve our some of our page objects actions and for that the main thing that we're going to do is have that explicit weight that we talked about in the section on weights so we're going to import from selenium webdriver.support we're going to import expected conditions we're going to call that ec expected conditions is a little bit of a long name to use when it's sprinkled in your code then from webdriver.support.ui we're going to import that webdriver weight object webdriver weight and then let's say we're going to import an exception because we're going to write a try accept in a bit what is this common common.exceptions import let's import no such no such element exception so we'll do that do that there and really the only place that we have to worry about is this this get element method so the first thing that I'm going to do is we're going to do that we're going to use a webdriver weight instead of just a find element for this you need the driver and that's on self.driver and then you need a wait time and there's actually in the in the settings file see how do I import this so from our settings file I'm going to import a wait time that's defined there and that's just so that if you do this webdriver weight method with expected conditions in various places you'll have a single definition for how long you want to wait that's kind of the way of doing things so if you go to the definition here the wait time is just a constant in seconds at 10 seconds that seems reasonable to me you can change it you can use a different integer there if you wanted that's just what I'm going to do to make things simple so we're going to wait until and now what are we waiting until expected condition let's say presence of element located on the page and we've got this locator type and then locator string and actually you have to I remember correctly I missed this up the first time you have to pass it as a tuple to this expected condition so we've got that element and then we can return it right and that's that so now we've got a slightly more sophisticated way of getting an element on a page which is that you'll wait for it to be there with this explicit wait so if we run the python search again so go there go to the programming language and then it'll tell us that the test passed right so we're still good our test still works like it did before it just has a slightly better way of handling handling waits in there now the last thing that I'm going to do is I'm going to let's put this in a try try accept block so if there's a no such element exception call that e then we'll print that what is this element with this is going to be a locator type of then locator string not found and then we can print that error which is e there so element with for example id of some id string not found and then it'll print the rest of the error there alright so now this is a pretty robust way of getting elements so let's save this run the run the test again run the search close it so we have the page there ran one test in about 8 seconds all good okay the test passed awesome so we've just written a testing module in selenium using the page object model design pattern and again let's go through what's going on you've got this main page locators class that just really has the tuples of how you find things on the page so something changes if the id changes all you have to do is go here change the string right or change the way you search for it and that's that you'll still be able to find that search input because what you're using in the page is just that that constant right you've already changed it here you don't have to go through all your code and try to figure out where things broke down you just change the search input our base page class here makes things really easy for us if we have another page that we want to test so if I wanted to do you know class advanced search page I could just inherit from the base page and write a new search method with the same stuff so self delete that self.enterText self.clickButton and then that's it I've just built out a search on a totally different page and all I have to do was add those two methods now of course I'd have to add locators and all that but that's pretty easy to do that's what building out that base page really gets us we've got the explicit weights here so if the element loads a little bit slower for whatever reason we're not going to run into an error like we saw in the section on weights we've got to try accept which you could put elsewhere I chose to put it here you could choose to put it elsewhere depending on your style and then we've got this nicely written test here that we didn't even have to touch so if things change on the page we don't have to touch this it's just doing a search and if you search for something you expect something back so you have a way of checking that all of this works really nicely together and then if there are other things you want to test on this page you can just add methods and build it out using that main page object and the methods on that object so hopefully this section was interesting to you guys I thought the page object model was really cool when I first learned about it and we're going to learn a bit more about it and practice it some more in our guided project so that will be the end for this section but we're going to get some more practice with it on our last guided project thanks everybody for joining and we'll see you in the next section hello everyone and welcome to our third guided project for this selenium course in this guided project we're going to be doing some more testing with the page object model that we covered in the previous section the goal of this project is to test the proper functioning of the band of booking form at thegoldbugs.com we need to verify two different things which is why we're going to be doing this testing we need to verify that our booking form does not submit without the required fields if you think about a real website you don't want to get information without the information that you're specifically looking for or the information that's required from some of your forms on your website and second we want to verify that the form does submit successfully when you fill it out with the required fields again if you can't recall from some of the previous sections where we've looked at this website which is a testing website for the course the booking form of interest to us looks a little like what's shown here we'll throw it up in a chrome browser in a later lecture as we get started with this but this is just so that you know what we're going to be taking a look at in this guided project we're going to use almost everything that we've learned so far in this course we're going to combine what we've learned about the page object model in the previous section with the form filler that we built for our second guided project by the end of this guided project you should be able to see the utility of the page object model in real life testing scenarios and we won't be starting from scratch in terms of the code like you'll see in the next lecture we're going to be starting with a little bit of a boiler plate so that we can get off to a running start and we'll see you in the next lecture alright so to get started on this project you should open up your vs code and then you can pull the boiler plate that we'll be starting from from our github repo so I'm just going to be doing this in my sandbox here which is just a folder for holding stuff that I'm working on but you can also go to the github and go to the guided project section and there's a guided projects that you can use to load directory where you can pull the starter code for this project so we'll have three python files the first one will be our test file so this is called testform we're importing unit test web driver service from our settings we're importing the chrome driver path in the goldbugs URL and then we've got this testing class so goldbugs main page form fill since that's where we're testing filling out the form unit test dot test case and then we've got a setup a setup method here that you should have seen before the one thing that we've added is self dot driver dot maximize window so that will maximize the window upon starting and then we've got a test method and a tear down method right which is just quitting the the chrome driver the web driver we've got a page dot py file this holds our base page class so we'll fill out these methods again and then we've got our main page class that inherits from base page and we'll fill that out that's related to that main goldbugs page and then we've got our main page locators class as well and this is to hold our locator tuples for the elements on the goldbugs page main page so what we might want to start with is let's actually start with the locators so we're going to be doing this form fill where we're filling out some required fields and some non-required fields and there might be some other elements that we'll need so I'm just going to have these three sections right here for now but let's actually open up this website in chrome so I've already got this open here and this is the website right so this is what we're going to be taking a look at and we saw this on the form filler so you scroll down and this is the section that we're interested in filling out so what we can do is go through this one by one again and take a look at all these elements and write up the tuples for how we find them and we're just going to have them in one place now okay so the required elements are we've got name email and then name, email and subject are required so we need these four text boxes are going to be required so let's click on them and start finding things out so this one the input name is fname so we'll do that and then I think last name is lname, yeah so last name is lname so we can just do that go back to vscode so first name we're going to have this all be uppercase so by we're going to be searching by name and this is fname and then last name is by.name lname okay and then we also need email and subject I think okay so now if we go back to chrome take a look at this, email I think isn't just name, email, yeah subject is this one id so let's just use that id control copy that go back over to vscode and paste that in so we're going to search here by. what are we searching for? by name email and this is going to be by.id and copy and paste that and I realize that red underscore here is telling me I need to import that class so from selenium.webdriver .common.by import that by class and then we'll be good to go except for this one which is misspelled it's about the lowercase there okay so we've got a required field locators control save that now let's go to our non-required fields and find those so let's head back over to chrome okay so keep scrolling down here now we're getting to the non-required elements I think this one if I remember correctly we could do it by id but let's just search by text area by tag name text area and then let's see we're going to say no response required that's just the label the input we'll search by an X path here I think so input with a value of no response required so let's go back over to VS code so we want this to be what let's call this message and call the other one no response required checkbox okay so the message we're going to search by tag name text area we're going to search by for the checkbox we're going to search by X path and this will be an input with a value equal to and then close that off value equal to no response required let's do that and let's keep going down this form here so the type of event we want the band to attend that'll be public show I think we can use an X path here again yeah we can just put what is this public show let's do public show radio button so this is going to be by X path the X path is basically the same as the one above it so this is input where the value what is that public show okay what else do we have we've got the select drop down right how did you hear about us and then we've got a survey element to fill out what is this survey radio button and I'll go back to chrome in a sec so let's go back I'm just flipping between these because it keeps the text a little bit larger in this instance rather than having them side by side just for now let's see so select drop down let's click on that select okay it's got that ID so we can copy that right and then I think for this one we're going to use the same X path so will you tell your friends about our band strongly agree this is input with a value of 2 okay so select drop down we're going to be doing this by by ID and that's what we copied so it's select and it's got that long string there and the survey radio button let's do this by X path this will be input with value of and you can be more specific here and say type radio if you wanted to but this works fine for now on our website and what else do we want I think that's going to be it for now so let's say that we're going to take a look at some other elements later on but that's going to be it for now so let's head over to our let's take a look at test form so what are we testing on this page we want to test an invalid form submission and a valid form submission so let's make this test invalid form submission and then we can make another method scroll this up so what are we calling this test valid form submission inheriting from self and this is just pass for now ok so let's go back up to this test invalid form submission what do we want to do with the with testing an invalid form submission so we're going to do what we did in the previous section so we start off with our main page that's going to inherit from the driver so we're going to have to import that so up here from our page file let's import that main page class and right now there's nothing in there, we'll fill that out as we go along so we've got the main page and that will inherit from the or rather take the driver as a parameter we're going to let me just take a look so let's actually take a step back go over to this form here when we pull up this page you pull it up here and you're taken to the top of the page so what we're going to want to do is let's do a clicking action so we can click on the booking form then we can fill out the non-required field so again we're testing an invalid form submission here so we'll fill out the non-required fields and then we'll submit so we can get the form as well the form element so that we can submit that in Selenium so let's go back over here to VS Code and write out those steps as methods that we'll then expand on in our main page class so we're going to click the booking form link that navigation header at the top of the page then we're going to we're going to fill the non-required fields then we're going to submit goldbugs form submit the form after we've filled everything out or filled out the non-required fields and then what we're going to do is assert is what should happen so let's go back to here if I click on this submit I haven't filled anything out I'm going to get some errors here so what we want to happen is we want to check that these these errors pop up there's this div I think here that will div with a class of field error that would tell us your form encountered a problem please scroll down to review we need that form to pop up so that the so that we know that the form has not been submitted correctly so assert is not none we need that form so let's do main page dot get form failure div because that's a div on the page and if we fail if that is none right if that if that method doesn't return anything we'll say alert for required fields is missing okay so that's the testing the invalid form submission now for the valid form submission we're going to basically do the same thing so we start off we'll initialize that main page object let's click on the booking form link again we haven't written this method we'll do that in a minute then we'll fill the required fields actually let's do a fill all fields not just the required fields so fill all fields so that'll include the required fields and then main page dot submit bugs form like we did before and then we'll do self assert is not none and then we're going to do essentially the opposite of what we've got here so get form failure div let's see what happens when we submit the form successfully so name whatever last name also whatever email john at do.com subject whatever and then we can hit submit and let's see what happens first name is not valid okay that's my bad it has to be a normal first name basically okay let's submit that and then we get this thank you so it says thank you for submitting the form let's click on that element form submission text class okay great so that'll be another thing that we'll have to look for but then we're going to assert is not none so we want that div to pop up with the with the thank you so we'll say main page get form success div and if that doesn't show up we'll say form was not submitted successfully right we've got an error because we did everything right and yet the form didn't submit we don't have that that expected behavior okay so there's a lot of stuff going on here now what we're going to do is fill this out in the page class so let's head over to the page class we've gotten the actions here in our test that we're going to want to have happen in our main page class so let's go ahead and fill these out so we need to click on the booking form link it's going to inherit from the self pass we'll fill that out in a later lecture we need to fill the required fields again I'm just creating the skeleton for this class here I'm not going to write out what all these methods are doing yet for that testing invalid form submission we're going to fill out the non-required fields then we're using it for the for the testing the valid submission and this one's going to be nice we can actually then just use fill non-required fields and fill required fields right because those two are mutually exclusive let's see what else do I have here we need to submit the form so we want to submit gold bugs form I want to let's pass keep going scroll down just give it more space then I need to get the form success and the form failure right so when the form doesn't submit we need to get that div so get form failure div and then get form was this form successive okay so those are all the methods so far that we need so these are all the methods that are related to our testing the other thing that I'm going to fill in on this base page so let's on this base page class we're going to have one for select by value because we have to use that drop down right so that's standard behavior so we want that in there and then I want to submit the form as well that's standard page behavior not specific to this gold buck page so we want that in our base page class not the main page class and now if I go back to the locators I think there's a couple more things I want to put in here so we want to submit the form so we need to get that form element we need to get the form failure div right for when you don't submit it successfully we need the form success div for when you do submit it successfully and then the booking oh yeah that's right I'm going to click on the I'm going to click on the booking form at first at the top of the page booking form anchor and so I need I need that as well so let's go back to let's go back to the gold bugs refresh this page just so that I can get the form there as well so let's see booking form let's click on that so we need something for that it doesn't seem like there's something good to identify it so let's just copy the expath go back to our locators here we're going to search by expath and then that's how we're going to search for it so you can copy that form element is the next one that we want so let's scroll down here let's try to find this form element click on this div is that in here okay yeah it is in here so the form and I think for this we can just search by tagging there's only one form on here so let's go back to VS code the form element we're going to search by I always forget that I have caps lock on search by tag name we're looking for that form element and then the failure div so let's submit this form has encountered a problem click on that I think we're looking for it by the class yeah so div.fielderror we'll do that so the form failure div this is just going to be we're going to look by CSS selector and this is a div.fielderror and then I think we can do the same thing for the success so fill this out again and then submit it john at do.com subject blah blah blah submit and then we've got to thank you, let's click on that so I know what that div class is it's got an ID so actually let's use that instead it's always better to be more specific when you can so it's got that ID let's use that as well so we'll do by ID and then copy and paste that so that should be good to go okay so we did a lot of stuff in this lecture so let's walk through kind of everything that we did so first thing we had our boilerplate code to start with so in our test we've got all the imports that we need we're going to run a setup method that sets up the web driver we're going to test the invalid form submission clicking on the form booking link to go to the form on the page filling on the non-required fields submitting the form and then making sure those failure dives show up to tell us that we need to fill in the information then we're going to test valid form submission so we click on the booking form link again fill in all the fields and submit the form and then check that that thank you div shows up as well so that's just we quit the driver when we're done testing on our page we'll fill this out in the next lecture I think we've got our main page class that inherits from the base page where it's got the specific actions for the gold bugs websites so filled required fields that specific to the gold bugs website and then we've got our base page class that holds all of those standard actions right get element, get text enter text, click stuff, select by value and then submit form that's all going to be in the base page class and then we took a look in Chrome at our web page layout right at the DOM and we collected all of these tuples in order to check in order to put them all in one place so that we can use them as our locators with these with these main page methods so that's what we're going to be working on in the next lectures that's it for now thanks everybody for joining and I'll see you in the next lecture welcome back everybody to this guided project we're going to keep going where we left off in the last lecture I know that it was a long one but the nice part about this lecture is that we'll only have to touch one python file so we've got our test set up here and so we don't need to touch that for now we've got our locators all collected here and so the only thing we're going to do is fill out this page object class that's really where we're going to be spending the bulk of our time this lecture so we've got this base page class so let's get started with filling this out so with git elements we want to get an element on the page what we're going to do is I'm going to return the self driver.find element and to find an element I need to have a locator type and a locator string this probably looks familiar to the last section so we're just going to keep rolling through here with this kind of stuff so git the elements we're going to find the element based on the locator type and the locator string next for our git text the element is going to be self.git element and we need that locator type and locator string so let's add those parameters here locator type locator string it's all well and good and then we can return element.text okay so we've gotten the text from a web element so enter text is next so for this we're going to input it into a text box right so we need to do what do we need to do here self.git element we need a locator type locator string add those parameters locator type locator string now that I've got the element I can send keys to it I'm going to send some text so I also need that as another parameter alright so that's the enter text method for the click we're going to get an element this probably looks straightforward by now so we need the locator type locator string locator type locator string and then we're just going to click on it and I'm not missing a parenthesis no not missing a parenthesis so everything's good alright and then the last one select or not the last one but the select by value so for this we're going to want that select object so we're going to have to import that and that comes so that comes up here don't collapse so let's import no this is from selenium dot webdriver dot support dot select you're going to import that select class alright so now we can instantiate that object and how do we do that well we do the self dot get element so we need those two locator type locator string locator type locator string so that instantiates the object and then what you want to do is what we saw in the form section so you can select by value and we need a value parameter so I'll add that up here ok and then finally we need to submit form so self dot get element add those parameters locator type locator string so that you can get the form and then you're just going to submit it right that's how forms work you can just call submit on the web element the form element and it'll submit that form for you ok so we've got this base page class here filled out now let's use that to put together the main page class and I think what we're going to do is we're actually going to wait for that for the next lecture for now so yeah let's do that and the last thing we'll do is we'll add some doc strings actually so we'll do the main page classes in the next lecture and just fill out the doc strings here this is select option in drop down menu based on value and then submit form submit form element right ok so I think that's it for now for this lecture in the next lecture we're going to use these base page methods that we've written out here to do the the main page methods down here so clicking on that booking form filling the required fields and all of that thanks everybody for joining and I'll see you in the next lecture hello everyone and welcome back to this um to this guided project so we're back in VS Code here where we left off in the last video so in this one again we're only going to be touching I think the page module so our testing modules all set up our locators are all fine now we're going to keep going with where we left off so we've got all these generic methods in our base page class so now let's start to hook them together in order to fill out this class for actions on the gold bugs home page so we'll just move down all these methods this excuse me skeleton that we put together before so for this let's see we're going to click on the we're going to do the doc strings as we go along as well so click on the booking form link to scroll to that section and how do we do this since we're inheriting we can just do self that click and then we need those locators right so now to go up here we want to import those locators add that so what am I importing from the locators import the main page locators class okay it's great we've got two spaces there so that's all well and good go back down here and then we can do what we saw in the previous section where we were testing that Wikipedia page right so what are we clicking on that booking form anchor so booking form anchor and then you do that star trick to unpack the the tuple so that's all we need for the clicking on the booking form it's fill the required fields so fill out only required fields in the form I was missing one so I want to enter text self enter text and we're going to do this going down the page so first name main page locators dot first name just make this John then you want to do the last name this is the last name make that do self dot enter text then we need the email so main page locators dot email John dot do at mail dot com not a real email address obviously at least I don't think it is and then we need the subject right subject love your band so that's what we wrote in the form filler that's what we're going to write here all right as you should be able to see this method reads in a very straightforward way because of this enter text that we've got from the base page class and then the way we're able to unpack these locators and then pass the kind of the testing parameters here right the text that we're going to be filling into these fields so let's keep going with with all the rest of this stuff so fill non-required fields so fill out only non-required fields in the form always miss the the end button on my keyboard so this is self we need to send a message right so main page locators dot message you're the greatest I love your band that's what we're going to say to them then what else do we have to do we have to click on something it's actually go back over to gold bugs to refresh this refresh my memory of what we're doing here filled out the required fields there filling in the message then we have to do these checkboxes the radio button the select and the the survey here that's what's left for us to do so we got to click on that checkbox right so we want to click on the main page locators dot no response required checkbox then we also have to click on the main page locators public show radio button right we want a public show and then we want we've got that drop down so main page locators the select drop down I think there's Google, Facebook maybe Instagram and others so we'll just select other and then we need to click on that radio button in the survey so main page locators survey radio button and that's all we need because we're just clicking on it okay great and then to fill out the all fields we're just going to do self. fill fill required fields and self.fill non-required fields right those are mutually exclusive so if we do them both in a row then they'll they'll all get filled out fill out both make this doc string here fill out both required and non required fields in the form okay and then finally we've got a couple more to fill out here submit submit the goldblocks form submit the booking form and this is just self.submit form we can pass it that locator right main page locators. form element that'll submit the form get form get form failure div this is get the div with the error message on failed form submission so for this we're just going to we want to return something right so we want to return self. get element main page locators and that's a form failure div that's the locator that we want and then we want to return the successive as well to check that the form was submitted successfully so main page locators that form successive and that let's add the the what is this the doc string so get the div with the thank you message on successful form submission okay so I think that's it for now let me take a look at this yeah so this is I think all we're going to do for now now we're actually at a place if you take a look at all this we're at a place where we can test it out there's one or two other things that I want to change here before we end this lecture but let's go back to our form test here and take a look at stuff and take a look at our locators too so locators we don't really need to do anything here we've got all the tuples that we need for this to work I believe we can test it out now so we can check and then we've got our testing class so it'll set up and it'll start the web driver go to that URL test the invalid form submission get that form failure on the page and throw an alert if we don't see that on an invalid form submission and then it'll quit that's how this set up teardown stuff works then it'll set up again test this valid form submission so click on the booking form link fill out all the fields submit the form and then check that you get that successive and the way this this is made so easy is through this inheritance here from the base page class that makes those low level interactions easy so that all we have to do in the main page class here is basically call those methods which we get through inheritance and then pass the locator tuples and maybe something else right like some text here if we want to enter some text but that makes this really pretty clean so now what we're going to do is let's run this actually let's run this test and see what happens here so let's run that in the terminal it should open this up right maximize it and then then it's just closed okay so it's happened failed errors too okay so we ran two tests and we failed them both so when we need to check on something so element not interactable okay so we've got some errors here and I think actually if you dig into this what you'll find I'm not going to go read through the error messages here well you'll find it's because we're not waiting for the page to load basically so we're looking for elements on the page that aren't there yet so what we need to do is we need to use those weights that we've seen before in the section on weights to make sure that we're actually waiting for the web elements to be selectable or clickable or what have you so we need to do some more import so from selenium.webdriver .support it's import expected conditions right we'll import that as easy then we've got to select what else do we have from selenium.webdriver .support .support .ui it's import we need to import that web driver weight and then let's from selenium let's do some try catches too or try accept common.exceptions it's import no such element exception and element click intercepted exception and you'll see where where we put that later on and then from our settings file it's import the wait time constant so I think that's just 10 seconds that's just so that we don't have constant sprinkled sprinkled everywhere so the first place we're going to wait is in this get element method so let's get rid of that and then start from scratch so we're going to do this try and then accept fill this out as we go now we want to use that web driver weight method that we saw right the explicit weight so web driver weight we're going to weight we're going to pass it self.driver and then some wait time until we're waiting until expected conditions presence of element located and we need the locator type and the locator string and that's all and once that's done if that runs then we want to return that element and that's how we get an element and then accept no such element exception if that's the case call that E and we'll say we'll just print a formatted string so an F string element with locator type of locator string not found and then print that error message too alright so that's how we'll do the get element there and then the other thing that we want to fix up is the click so we want to wait to make sure something is actually clickable when we do that when we try to click on it so let's try we're going to do web driver weight we don't need to return anything so I'm not going to store this as a as a variable self.driver then you need the wait time we're going to be waiting until expected condition and we need element to be clickable that's what we want to happen then we pass it that tuple of the locator what is this first it's locator type and then it's locator string so you're going to wait for that and then as soon as that's done you can click on it right since web driver weight we'll return that web element and then accept element click intercepted exception as e so that's if you can't click on it for whatever reason we're going to print f same type of error message so element click intercepted for locator type and of locator string and then print that print that error message so now let's give this another try and see what happens I think with these changes so with the additions of the explicit weights and the try accept blocks we should be able to run these tests successfully now so now let's run this it'll maximize the window and it's waiting so let's see let's see I'm going to take another look at this it's going to close down fail to read the scripture time out so two tests fail okay so let me take a quick look at this okay guys so I took a quick look as to why we're getting we're getting some errors running this and I realized there were a couple of the locators that needed to be changed so the two locators that we want to change are the booking form anchor so if you go to chrome let's go back over to chrome here where you're going to want to do is actually get the we can get the full x path it's not always the most ideal way to do this right but for now this is the way that we're going to do it because it works so the booking form anchor here you're still using x path but you've got this whole x path string here in order to get to that link and to click it and then the other one that you're going to want to change is the form successive so the ID I think the ID changes every time you put it in there every time you submit the form so it's not a reliable way of clicking on it or checking that it's there but what you can do we go back here and submit this form again is I know I get to do an email John at doh.com subject subject whatever and then submit the form let's do that thank you so if I click on that thank you it's got a class that we can use for now so form submission text is the class that we can use to find this because if you see the ID has changed from what it was before so that ID is not reliable in terms of testing right at least in terms of testing so if you change those ones that's the form successive and the booking form anchor to what you can get from Chrome let's go ahead and run this test again and we should be able to run successfully opens it up clicks on it goes down to the form fills it all out done right boom just completed the first test now this is running fills it out done submitted awesome ran two tests in 15 seconds okay no failures so that's great right I mean that's kind of what we want to see we just ran two tests in on our web page to check two different things that we wanted to validate we wanted to test that an invalid form submission failed which it did right based on the test that we wrote and we wanted to test that the valid sort form submission worked which it did based on the test that we wrote and all we had to do to kind of fix things up was to fix these locators that were wrong and then add these weights into the way we were clicking and getting elements on on a page so that's it for this lecture and in some sense we could stop there in the next lecture what I'm going to go through is just a few things that we can do to make it a little bit more self contained and improve our test just a little bit before we call it a done deal thanks everybody for joining and I'll see you in the next lecture hey everyone welcome back to our final guided project here where we're testing out the form filling on the gold bugs main page so what I'm going to want to do in this lecture is just improve slightly the way that we're testing this web page so I'm going to start by running the python files again to test the website and let's just think about what's going on here so it did one and now it's going to do the other okay so we ran our two tests and they ran successfully but let's think about what was going on and what we might want to do to improve it so at a basic level we're opening up this web driver and service and getting the URL twice right for both of our test for each one of them this setup and this teardown down here happen because these are test level it's in unit test so that means they run for each one of these tests but we're only testing one web page so if we want to make it better we might just want to have it only open up once and so you might be asking well is there a way to do that in unit test to only have this stuff run once so that we can maybe test the invalid form submission test the valid form submission by only opening up that web page once and not having to do it twice and the answer is yes there is actually a way of doing that and involves some class methods in unit test so what we're going to do is we're actually going to remove all of this stuff from our setup because we don't want to use this anymore this is just a test level setup method and as is teardown so teardown is a test level teardown method we don't want to use that what instead we want to use is we want to use a teardown class which is in unit test and then we'll do a setup class as well and for both of these to work you need to add a decorator calling them a class method so this is a little bit more of unit test than it is selenium specifically but it's still a nice way to make your selenium testing of your website slightly cleaner, slightly more you know advanced if you will or a little bit a little bit cleaner I guess is the good way to put it so this is called a class level setup method and then for this as well we have to add that decorator to call it a class method so let's start with the setup method for the class so we want to set the service the same way we did before so service executable path is just that chrome driver path and the driver is just going to be the web driver again right so web driver dot chrome and the service is the class dot service right that we just set up in this in this method then we can do class dot driver dot maximize window to maximize the window we can get the url class dot driver dot get then we need that goldbugs url and then the other thing we can do here is we can move up that instantiation of the main page so we can just say class dot main page since we're passing the driver we can just give it class dot driver right and then we can just call class dot main page click booking form link because that's part of that main page object so now we've got the class setup now let's do the class tear down so this is this is a class level tear down method and of course we want to quit the driver but now this is a class dot driver dot quit and this is just convention when you have a class method you usually say class instead of self but you could still say self if you wanted to it just makes it a little bit more more obvious so now let's take a look at our testing methods so by moving this main page up into the class setup we can get rid of the main page on both of these tests so we don't need to instantiate it anymore and we don't need to click on the booking form link anymore because we've already done that but what we do have to do is say this is a self this is now part of the class right so self dot main page get that that's how we want to change all of these references and you can do the same thing with the valid form submission with one exception being that we're going to call so we're testing the invalid form submission and filling out the non-required forms and then submitting the form but since we're not closing the window anymore what we can do is just fill out the required fields the method setup in our page class, our main page class so if you look at here we've got filled required fields filled non-required fields filled all fields right so we've already got that method written and then here like this self dot main page again self dot main page that is it let's add some let's add a doc string here because I've forgotten to add the doc string so test the submission of a form with required fields filled out and this is not a placeholder test method this is test the rejection of a form with required fields not filled out save that so again let's go through what's going on here we're using unit tests setup class method to only set up this private session and get the URL once and then to set up that main page object and get us down to the form once so that we don't have to do that twice or worry about the order of operations here so now let's run this and see what happens relative to what we had before so it opens up maximizes the window should get the URL so there we go gets the URL, clicks on it and there we go so we ran two tests and about the same time and yeah so you should see what you should have noticed there if we ran it again you'd see we're only opening up that Chrome browser once right we're not opening it up twice to run two different tests and so in that way our tests are a little bit cleaner a little bit more efficient again that's something of a unit test functionality then selenium functionality but it's a way of making your selenium test a lot cleaner and a lot easier to read and maintain okay thanks everybody for joining this one you should be proud of what you've done with this guided project right it's fairly large took a while to do so if we walk through what we did again we filled out all these locator tuples so we went through the web page to figure out where things are how we can locate them and then we stored them in this common class so that we could access them elsewhere we filled out our page object model so we started with the base page object expanded on it to use weights to get elements and built out these common web actions in that base page class we used those common actions like click and entering text to build specific actions for the goldbugs website right especially or specifically filling out that that booking form and then submitting it as well as getting getting the divs that we needed to get to check whether the the form was submitted successfully or not and then finally in our testing we initially had these test level setups with unit tests tested filling stuff out and making sure that it worked or didn't work as appropriately and we improved it a little bit by using the class methods and the class level setups and teardowns so that we're only opening that session once to do our tests of the goldbugs URL so a lot of stuff that went into that project if you made it this far you did a great job you've learned a lot hopefully in terms of how you can use Selenium to automate your testing or to scrape data from a web page but thanks everybody for joining this lecture and for going through this project