Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.


Set element wait

Many pages use Ajax technology, and the elements of the page are not loaded at the same time. To prevent errors from locating these elements while they are still loaded, you can set the elements to increase the stability of the script. Wait in WebDriver is classified into explicit wait and implicit wait.

Explicit waiting

Explicit wait: Set a timeout period, each period of time to check for the presence of the element, if it exists, the subsequent content is executed, if the maximum time (timeout) is exceeded, a TimeoutException is thrown. To display a wait, use WebDriverWait with either until or not until. Let’s talk about it in detail.

WebDriverWait (driver, a timeout, poll_frequency = 0.5, ignored_exceptions = None)

  • driver: Browser driver
  • timeout: Timeout period, in seconds
  • poll_frequency: Interval for each check. The default value is 0.5 seconds
  • ignored_exceptions: Specifies the exception to be ignored if calleduntil 或 until_notIs thrown in the process of the specified ignore exception, the code is not interrupted, the default ignore onlyNoSuchElementException 。

until(method, message=’ ‘) until_not(method, message=’ ‘)

  • method: Specifies the method to determine the expected condition, which is called at intervals during the wait period to determine whether the element exists until it appears.until_notConversely, when the element disappears or the specified condition fails, the subsequent code continues
  • message: If times out, it is thrownTimeoutExceptionAnd to displaymessageThe contents of the

The expected condition judgment method in method is provided by expected_conditions. The common methods are listed below.

Start by defining a locator

from selenium.webdriver.common.by import By from selenium import webdriver driver = webdriver.Chrome() locator = (By.ID,  'kw') element = driver.find_element_by_id('kw')Copy the code
methods describe
Title_is (‘ baidu ‘) Determine whether the title of the current page is as expected
Title_contains (‘ baidu) Determines whether the title of the current page contains the expected string
presence_of_element_located(locator) Determining whether an element has been added to the DOM tree does not necessarily mean that the element is visible
visibility_of_element_located(locator) Check whether the element is visible. Visible means the element is not hidden, and its width and height are not equal to 0
visibility_of(element) The previous method does the same thing, but takes an element as an argument
Text_to_be_present_in_element (locator, ‘baidu ‘) Determines whether the text in the element contains the expected string
Text_to_be_present_in_element_value (locator, ‘something ‘) Determines whether the value attribute in the element contains the expected string
frame_to_be_available_and_switch_to_it(locator) Check whether the frame can be switched in. If True, the frame can be switched in. If False, the frame can be switched in
invisibility_of_element_located(locator) Determines whether the element does not exist in the DOM tree or is not visible
element_to_be_clickable(locator) Determines if the element is visible and clickable
staleness_of(element) Wait for the element to be removed from the DOM tree
element_to_be_selected(element) To determine whether an element is selected, use a drop-down list
element_selection_state_to_be(element, True) Element, and the second argument is True/False
element_located_selection_state_to_be(locator, True) The preceding method has the same effect, but the locator parameter is passed in
alert_is_present() Check whether an alert exists on the page

Let’s write a simple example where we locate an element that doesn’t exist on the page and throw the exception we specified.

from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By driver = webdriver.Chrome() element = WebDriverWait(driver, 5, 0.5). Until (ec.presence_of_element_located ((by. ID, 'kw')), message=' timeout! ')Copy the code

An implicit wait

Implicit waiting also specifies a timeout, after which NoSuchElementException will be thrown if the specified element has not been loaded. In addition to the different exceptions that are thrown, implicit wait is global in that it does not affect code execution if the element can be located, but if it cannot be located, it polls the element until it is found, and throws an exception after the specified time.

Implicit waiting is much easier to implement than explicit waiting using implicitly_wait(). Example: Open your home page, set an implicit wait time of 5s, locate a non-existent element by ID, and finally print the thrown exception and runtime.

from selenium import webdriver from time import time driver = webdriver.Chrome() driver.get('https://blog.csdn.net/qq_43965708') start = time() driver.implicitly_wait(5) try: Driver.find_element_by_id ('kw') except Exception as e: print(e) print(f' time: {time()-start}')Copy the code

After the code runs to driver.find_element_by_id(‘kw’), the implicit wait is triggered and an exception is thrown if the element is still not located after 5 seconds of polling.

Mandatory waiting

Using time.sleep() to enforce waiting and set a fixed sleep time can have an impact on the efficiency of your code. As a reference, change the implicit wait to mandatory wait.

from selenium import webdriver from time import time, sleep driver = webdriver.Chrome() driver.get('https://blog.csdn.net/qq_43965708') start = time() sleep(5) try: Driver.find_element_by_id ('kw') except Exception as e: print(e) print(f' time: {time()-start}')Copy the code

It is worth noting that implicit wait is no different from forced wait in terms of time taken when elements cannot be located. But if the element is loaded after 2s, the implicit wait continues to execute the code below, but the sleep continues to wait for 3s.

Locate a set of elements

The previous article described 8 ways to locate an element. The method used to locate a group of elements is simply to change element to Elements, which is usually used to batch manipulate elements.

  • find_elements_by_id()
  • find_elements_by_name()
  • find_elements_by_class_name()
  • find_elements_by_tag_name()
  • find_elements_by_xpath()
  • find_elements_by_css_selector()
  • find_elements_by_link_text()
  • find_elements_by_partial_link_text()

Take the expert column of a blog on the CSDN homepage as an example.Use the followingfind_elements_by_xpathTo locate the names of three experts.This is the page code of the expert name part, I wonder if you have thought of how to passxpathWhat is the name of this group of experts?

From Selenium import webdriver # Set headless browser option = webdriver.chromeOptions () option.add_argument('--headless') driver = webdriver.Chrome(options=option) driver.get('https://blog.csdn.net/') p_list = driver.find_elements_by_xpath("//p[@class='name']") name = [p.text for p in p_list] nameCopy the code

Switching operation

Window switch

While Selenium is working on the page, it may be possible to click on a link and jump to a new page (a new TAB opens), but Selenium is actually on the previous page and needs to be switched to locate elements on the latest page.

Window switching requires the switch_to.windows() method.

First let’s look at the following code.

Code flow: Enter [CSDN homepage, save the handle to the current page, and then click on the left [CSDN official blogSkip to the new TAB page and save the handle to the page again. Let’s verifyseleniumWhether the system automatically locates the newly opened window.

From Selenium import WebDriver handles = [] driver = webdriver.chrome () driver.get('https://blog.csdn.net/') # Driver. implicitly_wait(3) # Obtain the current window handle (driver.current_window_handle) # click python, Driver.find_element_by_xpath ('//*[@id="mainContent"]/aside/div[1]/div').click() # Retrieve the handle to the current window Handle. Append (driver.window_handle) print(handles)Copy the code

You can see the first listhandleIt’s the same statementseleniumThe actual operation is still the CSDN home page without switching to the new page. Use the followingswitch_to.windows()To switch.

From Selenium import WebDriver handles = [] driver = webdriver.chrome () driver.get('https://blog.csdn.net/') # Driver. implicitly_wait(3) # Obtain the current window handle (driver.current_window_handle) # click python, Driver.find_element_by_xpath ('//*[@id="mainContent"]/aside/div[1]/div').click() # Switch window Driver.switch_to.window (driver.window_handles[-1]) print(handles) print(driver.window_handles)Copy the code

The code above is used after clicking the jumpswitch_toSwitch window,window_handlesThe returnedhandleThe list is sorted by the time the page appears, the latest page must be the lastdriver.window_handles[-1] + switch_toJump to the newly opened page.

If there are multiple open Windows, how to jump to the previously opened window? If there is such a requirement, the key(alias) and value(handle) of each window should be recorded to open the window, saved in the dictionary, and the handle can be retrieved according to the key.

Form switch

Many pages are also nested with a frame/ IFrame form. Selenium cannot directly locate these embedded pages. You need to use the switch_to.frame() method to switch the current object to the frame/iframe embedded page.

Switch_to.frame () can be located directly by default using the ID or name attribute, but if the iframe does not have an ID or name, then xpath is required to locate. Let’s first write a page containing iframe for testing.

<! DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta "> <title>Document</title> <style> div p {color: #red; animation: change 2s infinite; } @keyframes change { from { color: red; } to { color: blue; }} </style> </head> <body> <div> <p> python-sun</p> </div> <iframe src="https://blog.csdn.net/qq_43965708" width="400" height="200"></iframe> <! -- <iframe id="CSDN_info" name="Dream, Killer" SRC ="https://blog.csdn.net/qq_43965708" width="400" height="200"></iframe>  --> </body> </html>Copy the code

Now we can locate the CSDN button in the red box and jump to the CSDN home page.

From selenium import webdriver from pathlib import Path driver = webdriver.chrome () # get('file:///' + STR (Path(path.cwd (), 'iframe test.html ')) # 1. Locate driver.switch_to.frame('CSDN_info') # 2 by ID. Driver.switch_to. frame('Dream, Killer') # 3. Iframe_label = driver.find_element_by_xpath('/html/body/iframe') # driver.switch_to.frame(iframe_label) driver.find_element_by_xpath('//*[@id="csdn-toolbar"]/div/div/div[1]/div/a/img').click()Copy the code

Here are three ways to locate, all of which can be locatediframe 。

For those who are new to Python or want to learn Python, you can search “Python New Horizons” on wechat to communicate and learn with others. They are all beginners. Sometimes a simple question is stuck for a long time, but others may suddenly realize it with a little help.