Some of the problems with Web automation are that the front-end UI changes frequently, automation is slow, and code stability deteriorates as production environments change. So regular Web automation inevitably requires a lot of human cost to maintain, and the automation code is basically rewritten every time the requirements change or the front-end page is heavily refactored. In view of the above problems, we try to use automatic click traversal to realize automatic click traversal check of Web pages.

A background.

I have done automated tests on the Web front end (based on QT4W framework/selenium framework), and found that the front end controls need to be packaged separately. When the front end page controls are revised, this code basically cannot be used any more and needs to be re-packaged by myself. For demand, already developed into tests in a test environment to present a complete development of the UI controls, and if the front-end UI automation to do again, this time have the written code, write code and need time to debug, overall ROI is not ideal, even better than the manual testing takes more time.

Calculate ROI (return on Investment) according to the following formula: ROI= (manual test cost – automated test cost)/Automated test cost *100% Look at requirement 1. In the process of business test, some important pages (course details page, home page, classified search page and organization home page) often have some front-end UI optimization and adjustment. The test evaluation usually only evaluates the front-end UI changes that are included in the requirements sheet, and the test may only validate this.

However, for these important pages, the developers may make some minor changes themselves and do not test themselves, and do not inform the test students. There are some blind spot bugs in this area.

Look at two demand, develop students to optimize the front-end code, or change the online bugs, or added some js reporting and locate the log, don’t feel any problem issued free test (before they have a development on the modified mobile end bug when a debug code, lead to jump mobile version after the release of search agency home page will now blank). There are a lot of problems, so there is a need for the front end to automatically traverse and click through all the elements on the page.

Ii. Technical scheme

Technical points include: access to the browser driver permission, simulation link in the real browser opening situation, access to the page HTML code, traversal all a tag elements in the HTML code, multi-process to speed up traversal speed.

The first three issues can be implemented using the Selenium framework, while python has many libraries to support the latter two. So the overall project development can be completed quickly. The specific implementation idea is as follows:

Core code

3.1 Code Structure

1. Project Structure: Onlineedulib – Project library folder Course_detail. py – Encapsulate page Operation method Course_detail_element. py – Element encapsulate file pool.py – Process pool implementation file OnlineeDutest – Project Test Cases folder Course_detail. py – Page Test Cases (for Window System) Course_detail_linux. py – Page Test Case (for Linux system) settings.py – Configuration file requirements. TXT – Library file configuration Readme. md – Operation instruction file

3.2 Use case modules
# -*- coding: Utf-8 -*- "" Course Details page Test Case" "import sys import OS curPath = os.path.abspath(os.path.dirname(__file__)) rootPath = Os.path.split (curPath)[0] sys.path.append(rootPath) from Selenium import WebDriver # Import webDriver from Selenium Chrome import ChromeDriverManager # Import time from onlineedulib.course_detail Import Course_Detail # From Settings import URL, Class Course_Detail_Test(): "def run_test(self): Start = time.time() Driver = webdriver.Chrome(ChromeDriverManager().install()) driver.get(URL) # initialize your class object detail = Course_Detail() detail.get_page_a_label(driver, Deduplication () # delete detail.processing(driver) # delete detail.processing(driver) # delete detail.quit () # close the browser end = If __name__ == '__main__': Course_Detail_Test().run_test()Copy the code
3.3 Obtaining THE HTML code

Using selenium’s built-in ability to open a page in a browser based on the URL, you can retrieve the HTML code.

def get_page_a_label(self, driver = '', max_level = 1, traversal_level = 1): Body_element = driver.page_sourceCopy the code
3.4 Finding the A label using re

Import the python library re module for the re lookup, which is obtained according to the href property of the a tag. I need to replace it back and reprocess the tag list

Import re global cache_history_finall link_list = re.findall(r"(? <=href=\").+? (? = \ ") | (? <=href=\').+? (? =\')", body_element) # replace & convert to amp; Link_list: finall_link_list_sub1 = re.sub(r'amp; ', "", url) # delete for item in finall_link_list: if not item in finall_link: finall_link.append(item)Copy the code
3.5 Traversing storage
Traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level traversal_level cache_history_finall.append(url) elif traversal_level <= max_level and url not in cache_history_finall: cache_history_finall.append(url) print 'traversal_level:' + str(traversal_level) + ' ' + str(len(cache_history_finall)) + ' urlCopy the code
3.6 Recursive traversal
For url in finall_link: if traversal_level < max_level Self. exec_script('location.href="%s"' % url) print 'traversal_level:' + STR (traversal_level) +' +' + str(len(cache_history_finall)) + '+' + str(i) + ' url is:' + str(url) self.get_page_a(max_level, traversal_level + 1, url)Copy the code

Four, the practice

To facilitate testing and continuous delivery, I put the scripts on the server and need to modify the code slightly to execute Selenium scripts in Chrome on Linux.

1. Set traversal level to 1 and start to obtain page A tag link:

2. The acquisition is complete, 410 A tag links are obtained, and the execution time is 11s.

3. Assertion completion, 410 assertions, 426s execution time.

Five, the summary

Here is only the key module of the code, the specific code implementation details can see my GitHub project: github.com/1405942836/…