1, the introduction of

Packet Capture refers to the operation of capturing, resending, editing, and saving packets sent and received over the network.

In mobile development, it is necessary to monitor and test the network behavior of mobile phones and troubleshoot network and security-related problems. A packet capture function is launched on the Rock Mouse platform to facilitate users to quickly monitor the network. Android supports one-click packet capture, while iOS requires users to manually set the proxy to capture packets.

This paper mainly introduces the real machine packet capture scheme of rock mouse platform in mobile terminal. It has the following characteristics:

  • One-click open without any configuration (dynamically switch WIFI, set proxy, install certificate)

  • No need to rely on any analysis tools, support online real-time view of waterfall flow

  • Support for multiple protocols: HTTP/HTTP2 / HTTPS/WebSocket/TCP, etc

2. Effect demonstration

3. Packet capture principle

To realize the packet capture of App network data, it is necessary to monitor the network nodes between App and server, monitor any network node (network card), obtain all data passing through the network card, and parse these data according to network protocol, which is the basic principle of packet capture. However, the intermediate network nodes are not controlled by us, so packet capture cannot be realized basically, and packet capture can only be carried out on the client and server side. Usually we monitor the local nic data, as shown below:

Mobile phones and local networks are captured on the client side, while access devices and servers are captured on the background side. Local network refers to the route of WIFI. It is troublesome to directly capture the packet of router, so we add a layer of proxy service between mobile phone and local route, so that only the network data of proxy service can be captured, as shown below:

4. Scheme selection

Android does not support packet capture for security reasons. Therefore, there are many ways to capture packets on Android, but there are only two. One is to capture information using Tcpdump, which requires su users on Linux and root phones on Android. Tcpdump is a command line tool, which makes it difficult to analyze results. Another way is to capture packets by setting up network channels, such as network proxy, AP, etc. The advantage of this method is that mobile phone Root is not required, but the PC needs to support wireless network. However, Tcpdump does not support HTTPS, so we choose the second type.

4.1, Fiddler

Fiddler is originally a web debugging tool that listens for HTTP protocols, including HTTPS. His powerful function is to allow you to set breakpoints, modify the output data, and user interface friendly, simple to use, there are various data formats for parsing, convenient view. Another benefit of Fiddler is the ability to filter information from other programs by setting up proxies. Fiddler is based on Microsoft’s.NET platform, so it currently only supports Windos

4.2, Wireshark

Wireshark is the most popular packet capture tool. The Wireshark supports 850 protocols, ranging from the basic IP and DHCP protocols to advanced special protocols, such as AppleTalk and BitTorrent. Because Wireshark is open source software, new protocol support is added with each update. It is also a cross-platform software that can capture network protocol packets on Unix, Linux, MAC OS, Windows and other platforms, but it does not support Android yet. Wireshark, formerly known as Ethereal, changed its name to Wireshark in 2006 when its founder Gerald Combs jumped ship

4.3, Mitmproxy

Mitmproxy is an open source framework for man-in-the-middle proxies based on Python. At the same time, MitmProxy also has two associated components, one is MitmDump, which is the command line interface of MitmProxy. With it, we can connect Python script and use Python to realize the processing after listening. The other is MitmWeb, which is a Web application through which we can clearly observe the requests captured by MitmProxy.

4.4, Anyproxy

Anyproxy is an open source middleman proxy framework developed by Alibaba based on NodeJS. It supports HTTP/HTPS parsing and provides WEB interface

5. Comparison of schemes

Finally, we chose MitmProxy as the proxy service

6. Plan Evolution (Step pit tour)

From the general determination of the scheme to the final scheme landing, in the process of stepping on a lot of pits, the scheme has been adjusted for many times

Solution 1: PC + USB wireless NETWORK card open hot spot + mobile phone setting agent + Mitmproxy

The scheme is feasible but has some problems: Linux driver wireless network card is difficult; Wireless card hot spots are very unstable and often turn themselves off

Solution 2: PC + USB wireless NIC open hotspot + Iptables NAT + Mitmproxy

The scheme is feasible but has some problems: Linux driver wireless network card is difficult; Wireless network card open hot spot is very unstable, often will close by itself; NAT is only configured to transmit TCP through. UDP is cumbersome to configure

Solution 3: PC + dual wired NIC + wireless routing + Iptables NAT + Mitmproxy

The solution is feasible, but there are problems: Wireless routing requires the use of the wired network of the computer as the exit to access the Internet. The mobile phone connected to the wireless network opened under the router can only obtain the IP of the wireless router from Mitmproxy, but not the specific IP of the mobile phone. Therefore, Mitmproxy needs to be expanded

Solution 4: Server + wired NETWORK card + public hotspot (the device connected to the hotspot communicates with the server) + mobile phone setting proxy + Mitmproxy

Finally, plan 4 was landed on the Rock Rat platform

7. Data link

Process description:

1. The user clicks the “Open” button on the user interface of the real computer to enable the packet capture function

2. The UI layer sends the command to start capturing packets to the WEB layer through websocket

3. After receiving the command to enable packet capture, the WEB layer obtains the message content in websocket, installs the protocol agreed on the back-end of cloud real machine, and pushes the message to the message queue

4, cloud is the back-end to subscribe to the specified message in the message queue, when messages are received, calls the previously installed APP interface to switch on your mobile phone network and set up the agent operation (cloud true machine via a USB connection between back-end server and mobile phones, mobile phone pre-installed APP will launch a port for the cloud is the back-end call)

5. After receiving the request, the APP on the phone calls android SYSTEM API for network switching and proxy setting

6. After the mobile network is set successfully, the APP on the mobile phone can receive the message, and return the message one module by module through the original link, and finally return to the UI

7. After receiving the message, the UI invokes the packet capture service interface to start the packet capture service

8. After receiving the message, the packet capture service dynamically allocates a port to each mobile phone and returns the URL for viewing the waterfall stream online to the UI

9. After receiving the response result, the UI will embed the online viewing waterfall flow page in the form of IFrame, and the user can view the real-time request data

8. Special instructions

Android 7.0+ cannot capture APP HTTPS network packages by default due to system limitations. To capture the HTTPS network package of a specified APP, enable the network security configuration of the APP. For details, see

Developer.android.com/training/ar…

9, free trial free trial rock rat cloud real machine, experience the next key network capture package! Click to visit the Rock Mouse Cloud device platform