Introduction to the

Whistle is a cross-platform Web debugging agent based on Node. Similar to Fiddler and Charles, whistle is mainly used to view and modify HTTP, HTTPS, Websocket requests and responses. It can also be used as an HTTP proxy server.

After using Fiddler, Charles and Whistle, the advantages of Whistle are summarized as follows:

  1. Simple configuration: The configuration of whistle is similar to that of hosts in the system. All operations can be performed by configuring whistle. It supports multiple matching methods, such as domain name, path, regular expression, wildcard, and wildcard path.
  2. Support for extensions: Whistle provides plug-in extensions that allow you to add protocols to Whistle for more complex operations, store or monitor specific requests, integrate with a business local development and debugging environment, and do pretty much anything you want. It’s easy to develop, publish, and install whistle plug-ins.
  3. Built-in WEINRE: You can modify and debug mobile DOM structure and catch page exceptions through WeinRE.
  4. The interface is simple and easy to understand. From the interface, whistle functions are divided into four modules: Network, Rules, Values and Pulgins. You can switch between them by TAB.
  5. The documentation is comprehensive: Whistle’s official website provides detailed documentation for everything it can do at work.

Here’s what Whistle supports:

This paper will sort out the installation, startup, configuration and basic knowledge of Whisle based on the problems encountered in the process of using Whistle. When introducing installation, configuration, etc., you will sometimes mix up some problems you have encountered. If not, you can skip them. In addition, it should be noted that many contents that are not commonly used or rarely used will be very brief because they are combed in combination with user experience. If this article does not solve your problems, please go to the official website of Whsitle to check the documents.

Installing the

Install the Node

Since Whistle is node-based, you’ll need to install the Node environment first, so I won’t go into details here. Here’s what Whistle’s website suggests for the Node version:

Whistle supports Node versions v0.10.0 and older, and it is recommended to install the latest version of Node for better performance.

Install the whistle

After Node is installed, run the following command to install Whistle:

$ npm install -g whistle
Copy the code

View version:

whisle -V 
Copy the code

If the version information for Whistle is printed correctly, the installation is successful. You can then run the help command to view the help information:

whistle help
Copy the code

Start the whistle

Whistle supports three equivalent commands whistle, W2, and wProxy, and this article uses the W2 command. Here are some common commands:

Start the whistle:

w2 start
Copy the code

The default port for Whistle is 8899. To specify a port number, execute the following command:

w2 start -p 8888
Copy the code

Restart Whistle (also supports specified ports) :

w2 restart
Copy the code

Stop the whistle:

w2 stop
Copy the code

Configure the agent

There are two key parameters when configuring the proxy: the server IP and port number. The port number is simple, and corresponds to the port number started by the w2 start command. The server IP address can be set to either local (127.0.0.1) or remote (127.0.0.1). The console tells us the information needed for the configuration when we start Whistle, as shown below:

As you can see from the above picture, the port number is 8899 and the IP addresses are 127.0.0.1, 10.1.2.30, 192.168.137.1, and 192.168.191.1. If the IP address is a local proxy, all the four IP addresses can be configured. 127.0.0.1), if you are a remote proxy, you can use anything but 127.0.0.1 (10.1.2.30 is still not recommended for the same reason).

Here are several ways to configure a proxy:

Browser proxy

For browser proxy, you need to use the browser proxy plug-in. Here are two types: chrome and Firefox:

  1. Chrome: Use the SwitchOmega plugin.
  2. Firefox: Enter about: Preferences in the address bar, go to Network Proxy, select Manual Proxy Configuration, enter the Proxy server address and port, and save the Settings.

The global agent

  1. Windows: Menu > Settings > Network and Internet > Proxy

  1. Mac: System Preferences > Network > Advanced > Proxy > Web Proxy (HTTP) and Secure Web Proxy (HTTPS)

  1. Linux: I have not used it in my work. I will skip it here. More details on Whistle’s official website

Mobile agent

To configure the proxy on the mobile terminal, ensure that the wifi connected is on the same network as the host that has enabled the proxy. Take IOS as an example:

  1. Network connection (the company’s mobile phone and computer are not connected to the same network, which is a hot spot for the computer) :

  1. Click Configure Proxy and change to Manual:

Access the Whistle console

After configuring the proxy, verify whether the proxy is successfully configured. Unlike Fiddler and Charles, whistle requires a browser to access the configuration page (that is, the Whistle console) in two ways:

  1. Domain name access: local.whistlejs.com/
  2. IP + port access: for example, HTTP :192.168.191.1:8888

Since we are verifying that the configured proxy is in effect, the IP address for accessing the console should be the same as that for configuring the proxy.

If configured correctly, we should see whistle’s console as shown below:

If the page doesn’t open, the computer firewall on which Whistle is located may be blocking remote access to Whistle’s port, and you need to turn off the firewall or whitelist it.

To set the whitelist, perform the following steps:

  1. Control Panel -> System and Security -> Windows Defender Firewall -> Advanced Settings -> Inbound Rules -> New Rule
  2. Choose a port

3. Enter the port number set during wistle startup4. Select Allow the connection5. Enter a specification name6. Check the inbound rule list to see if it is successfully created and enabled. Once enabled, the whistle service can be accessed on the LAN

Installing the HTTPS Certificate

After turning off the firewall or whitelisting Whsitle, if whistle’s Settings page opens normally, it means we can proxy HTTP requests.

If your pages and interfaces are all HTTP requests, you can skip the STEP of installing HTTPS certificates. However, it is hard to find anything that is not HTTPS except for local or pre-delivery environments (many pre-delivery environments are HTTPS as well), so it is recommended to install the certificate in advance. If one of the following conditions occurs in your environment (which is almost always the case if the certificate is not installed), the HTTPS certificate is not installed or is not installed properly:

  1. The whisle configuration page can be opened, but the web page cannot be opened or only part of the page is loaded
  2. Jingdong App data cannot be updated or displayed incompletely, or “information cannot be obtained” is indicated by code scanning.
  3. The whistle configuration page has a small lock in front of the request for port 443 on the network, or the request cannot be caught
  4. “Your connection is not private”

Download the certificate

We can download the certificate in the following ways:

  1. The easiest way to download the proxy is to open a browser on the device where the proxy is configured and type Rootca. pro in the browser
  2. Open the configuration page in a browser on the machine that has started Whistle and click HTTPS to bring up an interface with a QR code. Click Download RootCA or scan the QR code to Download

Install the certificate

The installation of the certificate is independent of the agent tool, which means the steps are the same whether we’re using a Whistle, Fiddler, or Charles. I have summarized the installation steps separately, please refer to [Installation Certificate] for details.

Enabling HTTPS blocking

Once the certificate is installed, HTTPS blocking must be enabled before Whistle can see HTTPS and Websocket requests. Enabling HTTPS blocking and installing and trusting certificates are both necessary. We need to enable HTTPS blocking through the following steps: Open the configuration page in your browser on the machine that has whistle activated and click HTTPS. This will bring up a screen with a QR code and select Capture HTTPS CONNECTs.

Once the configuration is complete, if the HTTPS request still fails or a security alert appears, you can reopen your browser or restart Whistle. The whistle website explains that if you have visited the page before, the long link has already been established, so our configuration will not take effect.

basis

Although I’ll go into detail on how to use Whistle in specific scenarios in [Whistle Usage Practices], it’s important to have a quick look at the functional division of the Whistle console and how whisle is configured.

The console

The Way the Whistle console opens has already been covered and won’t be repeated here. The core of the Whistle console is partitioned as follows ([Whistle interface details click here])Wproxy.org/whistle/web…

  1. NetWork: displays the detailed response information and Timeline of the request list
  2. Rules: Matching Rules, whistle core, see configuration in the next section
  3. Values: specifies the key-value data. In Rules, you can obtain the value by using {key}
  4. Plugins: Displays the list of all installed plug-ins and enables or disables the plug-in function

Configuration mode

As mentioned at the beginning of this article, all whistle operations can be implemented through configuration. The configuration mode is extended to the system hosts configuration mode (IP domain or combination mode IP domain1 domain2 domainN), with richer matching modes and more flexible configuration mode.

The default configuration of a whistle is to write the pattern on the left and the operatorURI on the right. Whistle matches the requested URL with the pattern, and if it does, it performs the operation corresponding to operatorURI:

pattern operatorURI
Copy the code

Pattern and operatorURI can also be left and right interchangeably (in this case). For clarity and not to cause new confusion, I only introduce my common configuration methods here. I think it is enough to master only one.

When you configure hosts, if one IP address corresponds to multiple domain names, it is written as follows:

127.0. 01.  www.domain1.com www.domain2.com www.domainN.com
Copy the code

Like the system hosts, Whsitle supports the configuration of the combined mode if a pattern corresponds to multiple operations. When used in combination, whistle executes the operatorURI from left to right.

pattern operatorURI1 operatorURI2 operatorURIN
Copy the code

After a brief overview of the configuration, we can followpattern operatorURIMode for Whsitle added rules. Again, go back to the Whsitle console and select Rules. You can group rules in the same way you manage hosts with the SwitchHosts software. By Default, whistle has only one Default grouping, as follows:

Click the Create button to add a group of items to which all items related to the item can be added (to disable a configuration, use the Ctrl + / shortcut, or simply add #).

If you want the configured groups to take effect, you need to double-click TAB on the left page of the item. If a check appears, the configuration takes effect. If a group is not in use, it does not have a check.

Matching pattern

Whistle has the following matching modes:

  1. Domain name matching: A domain name can be matched with a specific domain name, port number, and protocol
// Matches all requests under the www.domain.com domain name, including HTTP, HTTPS, ws, WSS, and tunnel
www.domain.com operatorURI

// Matches HTTP requests under the www.domain.com domain name

http://www.domain.com operatorURI

// Matches requests for port 81 under the www.domain.com domain name (HTTP requests default to port 80, HTTPS requests default to port 443)
www.domain.com:81 operatorURI
Copy the code
  1. Path matching: Matches a path or defines the port number and protocol
/ www.domain.com:81/path/matching path and its subpaths (such as www.domain.com:81/path/child) request
www.domain.com:81/path operatorURI
Copy the code
  1. Exact match: Unlike path match, path match not only matches the corresponding path, but also matches the child path below the path. Exact match can only be specified by adding $before the path
// Matches the path of www.domain.com:81/path, excluding subpaths
$www.domain.com:81/path operatorURI
Copy the code
  1. /reg/ and /reg/ I ignore case and support sub-matching, but do not support /reg/g. In addition, you can pass part of the string in the request URL to the operatorURI through the sub-matching of the re
// Match all requests
* operatorURI

// Matches requests that contain the keyword in the URL, ignoring case
/keyword/i operatorURI

// Take the parameters in the URL to the matching action URI using a submatch
// The following re will take the file name in the request to the matching action URI
// Support up to 10 child matches $0... 9, where $0 represents the entire request URL, otherwise the same as the re's child match
/[^?#]\/([^\/]+)\.html/ protocol:/ /... The $1...
Copy the code
  1. Wildcard matching: In general, domain name and path matching can cover most of our needs. What’s not can be complemented by regular matches, but regular matches have a threshold for most of us, Whistle

Very thoughtful to provide us with a simpler wildcard matching method. I haven’t used wildcard matching yet, but here’s a quick introduction: Full wildcard matching: here

  • Wildcard match
// Start with ^
^www.example.com/test/*** protocol://... The $1... / / qualified end position ^ www.example.com/test/ * * * the test $protocol: / /... The $1...Copy the code

If the requested URL is www.example.com/test/abc?12… 1 = abc? 123&test, the second configuration 1 = ABC? 123, and www.example.com/test/abc?12. You can only match the first one.

  • The wildcard domain name matches
// Matches all urls ending in.com, such as test.com, abc.com, but not *.xxx.com
*.com protocol:/ /... The $1...
// Matches the subdomain of test.com, excluding test.com
// The domain name does not include the fourth-level domain names such as *.xxx.test.com. The domain name can only contain the third-level domain names such as a.test.com and www.test.com
*.test.com protocol:/ /... The $1...

// To enable all subdomains to take effect, use **
**.com protocol:/ /... The $1...

Copy the code
  • The wildcard path matches
// The path protocol:// A.B.C/XXX [/yyy] corresponding to all domain names takes effect* /127.0. 01.
Copy the code

The operating value operatorURI

Whistle’s official whistle website divides whsitle’s action values into strings and JSON objects. In this article, the operation values of whislte are divided into two types according to the configuration mode: those with and without Spaces.

  • Whitespace: STRINGS with whitespace and JSON objects with indentation reserved
  • Without Spaces: Strings without Spaces and serialized JSON objects without Spaces

Operation values without Spaces can be written directly to the operatorURI in the pattern opProtocol://(strValue). Note that the string must be wrapped in parentheses:

// Replace the return content of the URL that matches the pattern with helloWorld
pattern resBody://(helloworld)
Copy the code

Operation Values with Spaces need to be stored in Values or a local file.

  1. Save it in Values

In the Whsitle console, open the Values TAB, click Create, add the action value named test.json, and edit the contents of test.json on the right. Note that the value name is wrapped in parentheses as follows:

// Replace the returned contents of the URL that matches pattern with the contents of the test.json file
pattern resBody://{test.json}
Copy the code

The names of the added operation values are optional. If the suffix is used, it will be highlighted in the corresponding format. If not, it will be displayed in the default text format

  1. Save it in a local file

First, create a new test1.json file locally and click on the Files TAB in the Whsitle console

Select the test1.json file created by following the steps and Whsitle will generate a path that we can use

// Replace the returned contents of the URL that matches pattern with the contents of the test.json file
pattern resBody:///$whistle/test1.json
Copy the code