This article comes from the official official account of the project: “AirtestProject” copyright statement: reprint is allowed, but reprint must retain the original link; Do not use for commercial or illegal purposes

preface

Before a classmate said that he wanted to see the automation of Windows applications, so today we will use a simple example, take you to see what the automation of Windows applications have pits.

But before you do that, I’d like you to take a careful look at the “How to Connect to Windows Application Windows” tutorial, which describes in detail the various modes of connecting to Windows Windows.

Because of the diversification of desktop applications, the IDE to connect various Windows window will appear all sorts of problems, the students in the connection Windows window if there is a problem, such as can not connect embedded flash back, after connection, after the window is stretching and so on, suggest students to try several connection methods, will always find a good way to connect.

In addition, since students usually use mobile devices for testing, when switching to testing Windows applications, it is easy to apply some interfaces available only on mobile devices to Windows applications, resulting in interface failure and failing to reach the expected effect. We will also tell you in detail below to help you avoid pitfalls.

In field

(1) demand

Taking the Windows client of netease Cloud Music as an example, we can search for “Xue Zhiqian” in netease Cloud Music, then find his ranking in the playlist, add the top 10 songs to the list of “My favorite music”, then delete the content of the search box, and finally return to the home page of netease Cloud Music.

② The application window connecting netease Cloud Music in the IDE

To prevent other application Windows from interfering with the connection, we left only IDE window and netease Cloud Music window to be tested on the desktop. Then we select “Select Window” in Windows Window Connection, and then click to select the window of netease Cloud Music. After embedding, we found that the window of netease Cloud Music was severely stretched:

So let’s try another embedding method, the search window. Click “Search Window”, select the window of netease Cloud Music in the popup window list, and click “Connect” :

It is found that it can be embedded normally, so we can use the mode of search window to embed the Windows window of netease Cloud Music in the future.

Demand analysis and expected results
  • Click on the search box
  • Enter “Xue Zhiqian”
  • Enter xue Zhiqian’s song list
  • Play the top 10 songs in order of rank
  • Add the currently playing song to my Favorite music list
  • Click the search box again, select all the text and delete it
  • Return to the App Home Page

(4) Implementation of the code

Knowledge point analysis

① Simulate keyboard/mouse operation

The AirtestIDE supports testing of ordinary Windows applications by relying on the Image recognition framework (Airtest) for positioning, and using the Operation interface of PywinAuto for simulating operations.

The most common is to simulate keyboard/mouse operations. In the example above, we mainly used a simulated keyboard to help us type keywords, use shortcuts, and delete all items in the application.

Dev = device() # Select text(" ") dev = device() # select text(" ") dev = device() # select text(" ") dev. keyevent("{BACK}") dev.keyboard.SendKeys("^a") dev.keyboard.SendKeys("{DELETE}")Copy the code

{VK_CONTROL} = {VK_CONTROL}, so ‘^ a ‘means’ CTRL +a’. Pywinauto. Readthedocs. IO/en/latest/c… .

Emulating the mouse is not covered in the above examples, but the basic use is similar to the PywinAuto. keyboard module. Here’s a simple example of emulating the right click of the mouse:

dev = device()
dev.mouse.right_click(coords=(1180, 145))
Copy the code

Pywinauto. Mouse module can refer to this link: the content of the pywinauto. Readthedocs. IO/en/latest/c… .

2. Run keyevent on Windows applications

A lot of students have told us this before, why is it that when they enter various keyevents in Windows applications, they just enter the key code into the text box instead of the key code?

As shown in the figure above, we expected to delete the text of the input field, but the actual effect was to enter key codes into the input field one by one. The reason for this is that as we mentioned at the beginning of this article, this key code is applied on Android, and it is not applicable to Windows:

For Windows platform keys, we can refer to the pywinAuto. Keyboard module to give the key code collection:

③ A variety of screenshots

In the practical example, we used a number of screenshot statements to help us locate on netease Cloud Music. Here are a few screenshot tips to share with you:

First, when screenshots are taken, WE hope that we can try to reduce the amount of blank background. Too much blank background will easily lead to the scarcity of feature points, which will lead to the problem of incorrect location of screenshots.

Second, screenshots are not necessarily smaller and more accurate. When there are multiple screenshots of similar objects in the screen, we need to be flexible to add some auxiliary content to help us locate the target. For example, after typing “Xue Zhiqian”, we want to click on the second Xue Zhiqian in the drop-down list, but there are many disturbing elements in the list:

At this point, it would be unwise to sift through the screenshots with xue’s three characters in them, because it’s hard to be sure that the program will eventually match you to where on the list. But let’s try to change our thinking, appropriately expand the screenshot and add more feature points, so that our click target is still in the center of the screenshot, so that we can locate our target more accurately, right?

Finally, the clever use of target_pos helps us click on different locations in the screenshot. By default, the program will click on the center of the shot after finding the target on the screen, but in practice, we may need to click on the lower right corner of the shot, right of center, etc. In this case, we only need to set the value of target_pos (the value range is 1 to 9).

summary

Today’s tutorial content is here, if the students have what want to see the practical case, welcome to leave a message in the public account, in addition to the need to practice the script of the students, can also reply “netease Cloud” in the public account to obtain sample code.

Airtest website: airtest.netease.com/ Airtest tutorial website: airtest.doc.io.netease.com/ build enterprise private cloud service: airlab.163.com/b2b