This article is to give you an overview of what whistle can do. For details, see wproxy.org/whistle/

If you don’t already know what a whistle is, you can find out at GitHub: github.com/avwo/whistl…

directory

  1. Mode of operation
    • pattern
    • operation
    • filter
    • Other filtering methods
  2. Command line operation
  3. Plug-in extension
  4. Integrated whistle
  5. conclusion

Mode of operation

Almost all of whistle’s operations can be configured as follows (standard mode) :

pattern operation1 operation2 ... [filter1 filter2 ...]
Copy the code

Among them:

  1. Pattern: An expression that matches the requested URL. It can be a domain name, path, wildcard, or regular (more on this later).
  2. Operation: indicates the operation, consisting of protocol://opValue. Protocol indicates the operation type, and opValue indicates the specific operation, for example, reqCookies://(x-env=dev)
  3. Filter (Optional) : Pattern can only match the requested URL. To more accurately match the request (response) URL, method, IP address, status code, header, and content, you can use Filter. Filter includes excludeFilter and includeFilter

Such as:

Ke.qq.com 127.0.0.1:6001 reqCookies: / / (x - env = test) includeFilter: / / reqH: the user-agent = chrome excludeFilter: / / m: postCopy the code

If the request header contains chrome (case insensitive) string and is not a POST request, set host (127.0.0.1:6001) and add request Cookie (x-env=test).


The request was rejected because port 6001 was not enabled locally

If the first two urls of a rule are not urls, you can also configure the rule as follows:

Operation pattern1 pattern2... [filter1 filter2...].Copy the code

Operation and Pattern1 are not both urls

Compatibility mode can also be supported:

Operation1 operation2... Pattern1 pattern2... [filter1 filter2...].Copy the code

operation1 … OperationN cannot be a URL

Such as:

127.0.0.1:6001 includeFilter ke.qq.com fudao.qq.com: / / reqH: the user-agent = chrome excludeFilter: / / m: post resCors: / / * *. Url. Cn *. Alicdn.com 127.0.0.1:6001 reqCookies: / / ke.qq.com (x - env = test) fudao.qq.com includeFilter: / / reqH: the user-agent = chrome excludeFilter://m:postCopy the code

pattern

Pattern can be domain name, path, re, wildcard, etc. :

Domain name matching

  1. Matches the request for the specified domain name
    Ke.qq.com 127.0.0.1:6001 10.1.1.1:8080 *.url.cn *.alicdn.comCopy the code
  2. Matches the specified domain name and port number
    Ke.qq.com :5566 127.0.0.1:6001 # Match HTTP port 80 or HTTPS port 443 ke.qq.com/ 127.0.0.1:6001Copy the code
  3. Matches the specified protocol and domain name
    https://ke.qq.com 127.0.0.1:6001
    
    http://ke.qq.com 127.0.0.1:7001
    Copy the code

Path matching

Path matching is similar to domain name matching, but with more path fragments to match:

  1. Matches a request for the specified path
    Ke.qq.com/statics file:///User/xxx/test ke.qq.com/cgi-bin 10.1.1.1:8080Copy the code
  2. Matches the specified path and port
    file:///User/xxx/test ke.qq.com:8080/page fudao.qq.com/page/
    
    ke.qq.com:8080/cgi-bin 10.1.1.1
    Copy the code
  3. Matches the specified protocol and domain name
    file:///User/xxx/test fudao.qq.com/page/ 10.1.1.1 https://ke.qq.com:8080/cgi-bin https://ke.qq.com:8080/pageCopy the code

Regular match

Same as regular js expressions, it supports two modes: /regexp/ and /regexp/ I, and supports sub-matching. You can obtain the value of sub-matching in opValue:

  1. allke.qq.com/cgi-bin/And its subpathske.qq.com/cgi-bin/path/to?xxxIgnore all rules (equivalent to directly requesting the live network)
    /^\w+://ke\.qq\.com/cgi-bin\// ignore://*
    Copy the code
  2. containsimweb-testURL requests that are case-insensitive ignore all rules
    /imweb-test/i ignore://*
    Copy the code
  3. Delete md5 from URL and do this replacement
    /^\w+://\d\.url\.cn/path/to/(\w+)\.\w+\.\w+/ file:///User/test/xxx/dev/$1.$2
    Copy the code

Wildcard match

Regular expressions involve escape characters and exact matching. To solve these problems, Whistle provides wildcard matching to solve common matching problems:

  1. An exact match

    protocol://ke.qq.com; But does not include the subpaths $ke.qq.com file:///User/test/index.html # matches only at the request of http://ke.qq.com/test.html can contain the request parameters, But does not include the subpaths $http://ke.qq.com file:///User/test/index.htmlCopy the code
  2. The wildcard path matches

    # match /cgi-bin and its subpath /cgi-bin/path/to requests under all domain names (local replacement will automatically complete subsequent paths) */cgi-bin file:///User/test/Copy the code
  3. Normal wildcard matches (1 or consecutive * indicates a wildcard and can be used as a child match)

    ^*.url.cn/path/to/*.*.* file:///User/test/xxx/dev/$2.$3
    Copy the code

Domain name matching, path matching, and wildcard path matching have the advantage of automatically completing paths, that is:

*/cgi-bin file:///User/test/
Copy the code

https://ke.qq.com/cgi-bin/test.json will automatically match the local file /User/test/test.json. Other matching methods require sub-matching to complete the path itself. Wproxy.org/whistle/pat…

operation

Whistle abstracts each operation into a protocol. Each operation has a value: operation=protocol://opValue. Protocol indicates the operation type and opValue indicates the operation.

protocol

Whistle abstracts each type of operation into a protocol. If you change the request method (method:// POST) and change the headers (x-client=test&x-id=123), Cors (resCors://*)

  1. @function
  2. Setting UI Styles
  3. Set the hosts
  4. Set the agent
  5. Delay request
  6. Modifying the request URL
  7. Modify request method
  8. Modify the request header
  9. Delayed response
  10. Modifying the request content
  11. Modify the response status code
  12. Modify the response header
  13. Modifying a Response
  14. Request to replace
  15. Filter configuration
  16. Obtain captured packet data
  17. Dynamic setting rules
  18. Develop debugging tools

For details, see: wproxy.org/whistle/rul…

opValue

Each operation corresponds to an opValue, that is, a string or object. OpValue can be written in the following ways:

  1. Inline value:

    • Direct inlining (no whitespace, newline characters, etc.)
      # some rules that support strings, etc. Fudao.qq.com file://(hello) reqCookies://(x-env=test&x-id= ABC)Copy the code
    • Inline mode (support any character)
      ``` keyName1
      Hello world!
      -- 2020-1-1
      ```
      ``` keyName2
      x-env: test
      x-id: abc
      ```
      fudao.qq.com file://{keyName1} reqCookies://{keyName2}
      Copy the code

      Enclose inline values with three or more equal backquote pairs of + key names

  2. Values

    For large pairs of Values that are not suitable to be embedded directly into Rules, you can place them in Values at the same level as Rules.

  3. File or directory

    If some contents are too large to be placed on the WebUI, or you want to replace them locally, you can use the file or directory to mode:

    ke.qq.com/test file:///User/statics resCookies:///User/json/cookies.json
    Copy the code

    ke.qq.com/test and its subpath requests will automatically go to /Users/statics to find the corresponding file through the complete path and set the cookies in /User/json/cookies. Json

  4. Whistle provides a template string function similar to ES6. This function only supports inline Values or Values, not files or directories:

    ke.qq.com file://`(${query.test})`
    Copy the code

    Visit https://ke.qq.com?test=hello and return Hello

    ``` test.html
    url: ${url}
    ```
    ke.qq.com file://`{test.html}`
    Copy the code

    Visit https://ke.qq.com?test=hello to return to the URL: https://ke.qq.com?test=hello, or put it in Values

For details, see: wproxy.org/whistle/dat…

filter

Pattern can only match the REQUESTED URL. If the request method, client IP, request header, request content, response status code, and response are used for flexible and accurate matching, filter can be used. Filter includes includeFilter and excludeFilter, which are respectively used to realize the inclusion and exclusion matching rules. Any number of filters can be used for a rule:

ke.qq.com/cgi-bin resCors://enable resMerge://retcode=10000 includeFilter://resH:content-type=javascript includeFilter://resH:content-type=json excludeFilter://resH:content-type=gbk
Copy the code

If the content-Type of the response contains (case insensitive) javascript or JSON, If the value cannot contain GBK, resCors://enable and resMerge://retcode=10000 are executed.

If you think this line is too long, you can also write:

line`
ke.qq.com/cgi-bin resCors://enable resMerge://retcode=10000
includeFilter://resH:content-type=javascript
includeFilter://resH:content-type=json
excludeFilter://resH:content-type=gbk
`
Copy the code

When multiple includefilters or Excludefilters are mixed, includeFilter represents the relation of or, and excludeFilter represents the relation of and, that is:

pattern operation1 ... excludeFilter://p1 includeFilter://p2 excludeFilter://p3 includeFilter://p4 includeFilter://p5
Copy the code

The above request indicates that the URL pattern matches one of P2 or P4 or P5, but not P1 or P3.

Filter also supports regular expressions. For details, see wproxy.org/whistle/rul…

Other filtering methods

Mentioned above the filter only for special filtering, sometimes need to filter in other lines or configuration files possible rules, then the pattern can be used to ignore: / / protocol1 | protocol2 |… To filter the specified protocol:

^**/cgi-* ignore://* ^**/cgi-proxy ignore://! host ke.qq.com/index.html ignore://file|host www.baidu.com ignore://proxy|jsAppendCopy the code

Protocol :// XXX /cgi-yyy and its children ignore all configuration rules, equivalent to no rules (but protocol:// XXX /cgi-proxy and its children retain the host rule); The request for ke.qq.com/index.html ignores local substitution and host rules; All requests for www.baidu.com ignore the proxy and the rule to append jsAppend.

For details about ignore, see wproxy.org/whistle/rul…

In addition to filter and ignore, you can use scripts or plug-ins to implement more complex matching methods. See:

  1. ReqScript:wproxy.org/whistle/rul…
  2. ResScript:wproxy.org/whistle/rul…
  3. Plug-in: wproxy.org/whistle/plu…

Command line operation

  1. w2 add: of the current directory.whistle.jsFile to set project rules. For details, see:Wproxy.org/whistle/cli…
  2. w2 i whistle.xxxInstall the Whislte plug-in (plug-ins can also be installed globally using NPM)

    You can also specify the registy installation plug-in: w2 I whistle. XXX –registry=http://xxx, or install it directly with an internal command such as TNPM: w2 ti whistle.xxx

  3. w2 uninstall whistle.xxx: This uninstallation mode supports only throughw2 [x]i whistle.xxxInstalled plug-ins
  4. w2 run xxx: Execution passedw2 iInstalled plug-ins come with commands (not globally installed, so cannot be executed directly)
  5. Whistle also supports boot arguments or command-line arguments for special scenarios, see:w2 help

Plug-in extension

Whistle provides a powerful and comprehensive operating protocol and supports customization of its own operating protocol through plug-ins. Each plug-in is an NPM package that not only has the extensibility provided by Whistle, but also has all the capabilities of Node.

  1. Provide UI interface
  2. As the request Server
  3. Statistics request information (view report/log data, etc.)
  4. Set rules (dynamic, static, global, and private rules)
  5. Obtain captured packet data
  6. Codec request response data stream (Pipe Stream feature)
  7. Extended Network right-click menu
  8. Synchronize Rules and Values data

How to develop, debug, publish, and use plug-ins is not described here, just refer to the documentation: wproxy.org/whistle/plu…

Integrated whistle

Whistle not only supports plugin extensions, but can also be packaged as a normal NPM package with a plugin for third-party applications. For details, see github.com/nohosts/noh…

conclusion

This article briefly explains the pattern Operation filter (operation=protocol://opValue), command-line operations, plug-in extensions, and how to integrate whistle into your application. In addition, whistle has many functions that are hard to list in a single article. If you have any questions or suggestions, you can give feedback via GitHub issue: github.com/avwo/whistl…