Charles profile
Charles is a common network packet interception tool for PCS, but also for mobile network requests. In addition to the debug interface, Charles can also be used to analyze requests from third-party applications. In conjunction with Charles’ SSL capabilities, Charles can also analyze the Https protocol. Charles supports the following protocols:
- HTTP / 1.1
- HTTPS
- HTTP/2
- Ws (WebSocket)
- WebSocket Secure (WSS) SOCKS
Charles is a paid software with a 30-day free trial. After the trial period, users who have not paid will still be able to use the app for no more than 30 minutes at a time, and there will be a 10-second delay on startup. Therefore, the payment plan is quite user friendly, even if you don’t pay for a long time, you can still use the full software features. However, when you need to do packet debugging for a long time, you will be affected by Charles forcing the shutdown.
Charels official website and documentation
- www.charlesproxy.com/
- www.charlesproxy.com/documentati…
Charles Packet capture principle
- Man-in-the-middle attack (MITM)
In the field of cryptography and computer security, a man-in-the-middle attack means that the attacker establishes an independent contact with the two ends of the communication and exchanges the data they receive, so that the two ends of the communication think they are talking to each other directly through a private connection, but in fact the whole conversation is completely controlled by the attacker.
Principle of HTTP packet capture
- The browser (Client) makes a request from port number 56075, and the request is sent to port 8888 (MITM Server) on which local Charles listens. The connection is made directly on the local machine
- After receiving the browser request, Charles makes a new request from port number 56076 (MITM Client). Since the network packet needs to be connected to the public network, the IP address is 192.168.31.44 (my computer’S IP address, described in the next section). The IP address of cdn.Staticfile.org (Server) is 111.63.183.223, and the port number is 80 because it is an HTTP request
- 111.63.183.223:80 returns an HTTP response to Charles’s proxy client at 192.168.31.44:56076
- Charles does some internal processing (Capture & Analysis) and sends the response to 127.0.0.1:56075 through port 8888, where the browser receives the response
HTTPS process
- The client initiates an HTTPS request to connect to port 443 of the server
- After receiving the request, the server sends the certificate and public key to the client
- The client parses the certificate and, if there is no problem, uses an encryption algorithm to generate a symmetric key, which is then encrypted with the public key
- Send the encrypted key + data to the server
- The server decrypts the private key to obtain the symmetric key and then communicates symmetrically
Principle of HTTPS packet capture
Looks pretty secure, but how does a man-in-the-middle attack work?
- When the client initiates an SSL handshake, the middleman hijacks the user request and disguises it as the client initiates an SSL handshake.
- The server sends the public key to the middleman, who captures the public key, saves it, and sends its own public key to the client.
- The client obtains the replaced public key and encrypts the symmetric key of the client and sends a request to the middleman. The middleman uses his private key to obtain the symmetric key. The middleman then encrypts the symmetric key with the public key sent from the server and sends the request to the server.
- Then all the data requested by the client and server is decrypted by the middleman using symmetric keys to get all the information.
So to capture HTTPS, we need to install and trust its certificate.
Install the certificate
Select the menu Help -> SSL Proxying
- Install Charles Root Certificate: The PC installs the Charles Certificate and trusts the Certificate
- Install Charles Root Certificate On a Mobile Device or Remote Browser: You need to Install and trust Charles Certificate to capture packets On Mobile devices
The PC captures packets
- Choose Menu Proxy -> Select macOS Proxy
- Click On Proxy Settings and configure it as shown in the following two images
Packet capture on the mobile terminal
Select the wifi of the mobile device to go to the configuration page, set the IP address to the IP address of the PC in the unified LAN environment, and Charles listening port 8888.
Why some tripartite applications can not capture packets
Certificate Pinning
Certificate Pinning is when the client has the true public key certificates of the server built into it.
In an HTTPS request, the public key certificate sent by the server to the client must be the same as the built-in public key certificate on the client. Generally, companies that pay more attention to safety will adopt this operation.
In this case, when Charles is used to capture packets, the public key certificate of Charles is different from the public key certificate of the client. The forged request will be rejected, and the packet capture will fail. So what’s the solution to this situation?
- Hack way, brush ROOT or jailbreak, use tools to remove the fixed public key certificate in APP;
- The other is the right way, you have the development permission of this APP, then you generally have the public key certificate and the accompanying private key, we can import the certificate and private key into Charles, to solve the problem caused by the fixed certificate.
Charles To import the public key Certificate and private key, click Charles -> Proxy -> SSL Proxying Setting -> Root Certificate, and then import the. Pem or p12 file.
Certificate two-way authentication
In most cases, the client authenticates the server for TLS, but in some security-sensitive scenarios (such as anonymous social networking), some apps enable two-way authentication for TLS, which means that the server also authenticates the client.
- In this case, the client has a built-in set of public key certificates and private keys. Compared with the server side, APP has a higher risk of shell cracking, so the public key certificate and private key are usually extremely hidden, such as written to the library, hidden in a confused random number algorithm function, which makes it more difficult to decode.
- Nuggets App has a built-in certificate, so it can’t capture packets, but its Web side can capture packets normally.
Modify HTTP requests and responses
For a qualified request, modifying its header, parmater, path, host, query, and Body can also modify the result of the response.
Protobuf serialization
With a Protobuf, the default return is the number column as the key, which is not intuitive, so it is impossible to know exactly what this data means.
Select View -> Protobuf Settings in Charles and add proto file to Charles
The resources
Q: Why did your Charles fail to catch the bag?