This article from the official public project number: AirtestProject links: the original airtest.doc.io.netease.com/tutorial/7_… Copyright notice: permission to reprint, but reprint must retain the original link

preface

This tutorial explains how to use the AirtestIDE to test a Windows application automatically (taking netease Cloud Music Windows as an example).

By reading this article, you will learn:

  • Use the AirtestIDE to record a script of a Windows software window
  • How to invoke the interface of Windows operation
  • How to specify a connection to a Windows window in a script and command line

1. Fundamentals

The AirtestIDE supports testing of common Windows applications by positioning themselves using the image recognition framework (Airtest) and using the operation interface of PywinAuto to simulate the operation.

Poco support for Windows is not yet available, but we’ll be rolling it out soon. Stay tuned.

2. Connect a Windows window in the AirtestIDE

To test a Windows program by using the AirtestIDE, first connect the Windows window to be tested with the AirtestIDE, that is, tell the handle of the Windows window to the AirtestIDE. After successfully connecting to a window, Airtest can treat the window as a device and then perform image recognition, click actions, and more on the Windows window as if it were operating on an Android device.

Note: Since you have selected a Windows window to connect to, you will rely on this specific window handle to operate and run the script in the AirtestIDE. Therefore, after the window is selected, please do not close it at will. Otherwise, an error will be reported because the corresponding window cannot be found during runtime.

Conventional connection method – one – key embedding

The AirtestIDE provides one-click embedding into a Windows window, as shown below:

By clicking the selected window button of the AirtestIDE, you can select a Windows app already launched on your desktop (the window will display a green box at the edge) and embed it into the AirtestIDE. After embedding a window, you can easily operate a window, record a statement, and run a script in the AirtestIDE.

Another alternative connection method

The underlying implementation of Windows applications is different. If you use the default one-click embedding method to embed Windows into the AirtestIDE, you will encounter some problems (for example, you can’t operate the mouse, you can’t embed Windows properly, you can’t display images after embedding Windows).

To solve these problems, we have provided a device-Windows Embed Backup Method without Embed connection. Please find the Device-Windows Embed Backup Method in the Settings panel of AirtestIDE and check the check box to connect the window.

The next connection method is the same as the normal method, click the selected window, then select the program under test window:

As you can see from the picture, this window connection method does not embed a Windows window into the AirtestIDE. Although it is not as simple and effective as the default method when recording a script, it can avoid some problems caused by window embedding.

The desktop model

If you want to test more than one window (with multiple different window handles), embedding a single window may not be sufficient for testing, we also provide a third solution: desktop mode.

Click the desktop mode button and the device window of the AirtestIDE will be completely hidden to enter the Windows desktop mode.

In this mode, the recording script and running script are the same as normal. When you execute the script, you can identify a screenshot of the entire desktop and even a screenshot in the code window of the AirtestIDE. Avoid screenshots in the script interface interfering with execution results.

Connection Precautions

  • Due to the selected window, the table may have many other Windows open at the same time, leads green box of choice will not be able to accurately choose the application under test, please before the selected window click on the button, as far as possible to minimize all other irrelevant window first, such doing can let Windows choose more accurate.
  • To prevent exceptions, do not select your own desktop and AirtestIDE software ontology. Because window embedding works by setting up a Windows window as a child of the IDE, trying to connect to the desktop can cause unexpected exceptions. If you just want to test all the Windows on your desktop, you can choose our desktop mode to record the script.
  • Since Windows Windows need to record resolution for screenshots, embedding the window into the IDE will be fixed in size and cannot be changed. If the embedded window is too large, you can adjust the window to an appropriate size before embedding it.

Exiting a Windows connection

  • To exit the Windows connection by directly closing the AirtestIDE, you can automatically pop up the embedded window
  • You can also go through the top right cornerDisconnect current deviceButton to exit the window connection

3. Record and write Airtest scripts

After successfully connecting to the Windows window, it is equivalent to we have connected to a device, and then it can record and play back the script.

Generate a screenshot statement

First, the touch/wait/EXISTS/ASSERt_EXISTS/ASSERt_not_EXISTS statements with screenshots are not much different from those recorded on an Android/iOS device:

However, it should be noted that after selecting the desired position with the mouse drag box, you need to double click the screenshot area to complete the screenshot, instead of automatically completing the screenshot when the mouse is released. This is also the biggest difference of the screenshot under Windows.

So the steps for the screenshot are:

  • Drag the mouse box to select the appropriate area
  • Double-click the mouse to complete the screenshot
  • Click the right mouse button to re-select the box
  • According to theEscButton to exit the screenshot operation

Also, when recording the SWIPE statement, after selecting the screenshot area, you need to click the swipe end point to record the statement.

Calling Windows Interfaces

Like Android/iOS, Airtest encapsulates common operations on Windows, using the PyWinAuto library as the underlying interface.

Therefore, when writing test scripts for Windows applications, we may want to consult the following API documentation:

  • Airtest cross-platform API, here all API can be directly called in the script
  • Airtest’s Windows-specific API, please refer to the code examples provided below to see how to call Windows device-specific API
  • If you need more complex operations, you may want to consult the API provided by PyWinAuto

A simple example

For demonstration purposes, let’s assume that instead of passing in arguments on the command line, the script uses the connect_device interface to attach a window with handle 123456 and do something with it:

from airtest.core.api import *
dev = connect_device("Windows:///123456")
# Universal interface call, same as other platforms:The touch (picture)Copy the code

Let’s say we want to be able to call some Windows-specific operations. For example, after reviewing Airtest’s Windows-specific API documentation, we find that there are some operations that are only available for Windows:

Call a Windows-specific interface, such as getting the title content of the current window
print(dev.get_title())
# Move the window to a coordinate position
dev.move((100, 200))
Copy the code

Next, suppose we want to use the mouse wheel now. After checking the API of Airtest, we find that the Windows module of Airtest does not encapsulate the function of the mouse wheel. At this time, we can further consult the documentation of PyWinAuto and find the chapter related to mouse. How to call the mouse wheel interface:

dev.mouse.scroll(coords=(80, 100), wheel_dist=1)
Copy the code

Input keyevent

On Android, you can press the HOME button through keyevent(“HOME”). On Windows, you can also send keystrokes through the Keyevent interface. The Android keycode is based on ADB, while the Windows module of Airtest encapsulates the keycode supported by PyWinAuto. Please refer to the pyWinAuto. Keyboard documentation to write keyevent interface parameters under Windows:

# In PyWinAuto, the symbol ^ also represents the CTRL key, so ^a is all (CTRL +A).
keyevent("^a")
Delete key = delete key = delete key
keyevent("{DEL}")
Copy the code

Please write the code of key response after consulting the document according to the actual requirements.

4. How do I specify a window to connect to when running the script

Running scripts and viewing reports in Windows mode of AirtestIDE is no different from any other platform:

The most important thing to note, however, is that any window connected to the AirtestIDE is connected using a window handle. A window handle is a unique 32-bit unsigned integer owned by each Windows window object, and this value changes each time the window is opened.

This means that if you write a script embedded in the AirtestIDE using a window, you can run it this time, but copying the command line of the script in the AirtestIDE can’t guarantee that it will run again. The command line of the AirtestIDE will have the parameter device Windows:/// handle, and the next time you open the window, the handle will have changed.

Therefore, we can also connect Windows in other ways without using handles. Airtest’s Windows module uses PyWinAuto’s Connect interface to connect Windows, so in addition to the handle, we can use the window title to connect:

Connect to a Windows window with handle 123456
Windows:///123456
# connect a Windows window whose name matches a regular expressionWindows:///? title_re=Unity.*# Connect to Windows desktop, do not specify any window, corresponding to the IDE desktop mode
Windows:///
Copy the code

If you want to use window name concatenation on the command line, you do not need to add quotation marks or other symbols, just write:

airtest run test.air --device Windows:///? title_re=Unity.*Copy the code

5. Poco support

Access the Windows window of the POco-SDK

There are some Windows-apps that have Poco access (such as the Unity game window), and we can also use the AirtestIDE to view and select the UI node easily. To use the function, you need to tell the AirtestIDE the location and size of the Windows window in advance.

From “Options” – “Settings” – “Poco”, select “Windows Content Area Rect”, select the main screen of the window, double click to confirm (right click/ESC to cancel), and then start to use Poco related functions, as shown in the picture:

Poco – Windows support

We are working on documentation and features for Poco support for Windows, so stay tuned.

For more information about the tutorials and the project, please visit our official account AirtestProject to check out the previous tutorials: