In the process of software development, we will encounter a lot of places that need proxies, such as packet capture, HTTP content transmission, such as Nginx reverse proxy.

In Linux, a Privoxy Socket proxy is installed to convert HTTP proxies. However, Privoxy was difficult to use on Mac using Brew, so I wanted to try a program that handled sockets and HTTP proxies without having to install a separate program to do the conversion.

I haven’t done much network programming before. Recently, I just happened to be studying Go and practicing my skills.

Here we mainly talk about the use of HTTP / 1.1 protocol in the CONNECT method to establish a tunnel connection, the implementation of HTTP Proxy. The advantage of this proxy is that you do not need to know the data requested by the client, just need to forward it intact, for processing HTTPS requests is very convenient, do not have to parse his content, you can achieve proxy.

Start proxy listening

To make an HTTP Proxy, we need to start a server that listens on a port to receive requests from clients. Golang gives us a powerful NET package to use, and it is very easy to start a proxy server listening.

l, err := net.Listen("tcp", ":8080") if err ! = nil { log.Panic(err) }Copy the code

The above proxy we implement a server listening on port 8080, we do not write IP address here, default listening on all IP addresses. If you only want to use 127.0.0.1:8080 locally, then the machine can’t access your proxy server.

Listen to receive proxy requests

Once the proxy server is started, we can begin to fail to accept proxy requests, and then we can proceed with further processing.

for { client, err := l.Accept() if err ! = nil { log.Panic(err) } go handleClientRequest(client) }Copy the code

The Accept method of the Listener interface, which receives connection data from the client, is a blocking method. If the client does not receive connection data, it blocks and waits. The received connection data is immediately handed to the handleClientRequest method for processing. The purpose of opening a goroutine with the go keyword is not to block the client’s reception, so that the proxy server can immediately receive the next connection request.

Parse the request to get the IP and port to access

With the client proxy request, we also have to extract from the request client to access the remote host IP and port, so that our proxy server can establish a connection with the remote host, proxy forwarding.

The HTTP header contains the host name (IP) and port information we need, and it is in plain text. The protocol is very standard, similar to:

CONNECT www.google.com:443 HTTP/1.1 Host: www.google.com:443 proxy-connection: keep-alive user-agent: Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36Copy the code

You can see that what we need is on the first line, the first line information is separated by Spaces, the first part CONNECT is the request method, this is CONNECT, and then GET, POST, and so on, which are all standard HTTP methods.

The second part is the URL, the HTTPS request is only host and port, the HTTP request is a completed URL, we’ll see an example in a second, we’ll see that.

The third part is the HTTP protocol and version, which we don’t have to pay too much attention to.

This is an HTTPS request. Let’s look at HTTP:

GET http://www.flysnow.org/ HTTP/1.1 Host: www.flysnow.org proxy-connection: keep-alive upgrade-insecure Requests: 1 the user-agent: Mozilla / 5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36Copy the code

You can see HTT without port number (default 80); More than HTTPS schame – http://.

With this analysis, we can now get the request URL and Method information from the HTTP header.

var b [1024]byte n, err := client.Read(b[:]) if err ! = nil { log.Println(err) return } var method, host, address string fmt.Sscanf(string(b[:bytes.IndexByte(b[:], '\n')]), "%s%s", &method, &host) hostPortURL, err := url.Parse(host) if err ! = nil { log.Println(err) return }Copy the code

Then we need to further parse the URL to get the remote server information we need

If hostPorturl.opaque == "443" {// HTTPS access address = hostPorturl.scheme + ":443"} else {// HTTP access if Strings. Index(hostPortURL.Host, ":") == -1 {// Host does not have a port, Default 80 address = hostPorturl.host + ":80"} else {address = hostPorturl.host}}Copy the code

This completes the retrieval of the requested server’s information, which may be in one of the following formats

ip:port
hostname:port
domainname:portCopy the code

It could be IP (v4ORv6), it could be host name (Intranet), it could be domain name (DNS resolution)

The proxy server establishes a connection with the remote server

With the information of the remote server, you can dial up to establish a connection. With the connection, you can communicate.

Server, err := net.Dial(" TCP ", address) if err! = nil { log.Println(err) return }Copy the code

Data forwarding

After successful dial-up, data proxy transmission can be performed

if method == "CONNECT" { fmt.Fprint(client, "HTTP/1.1 200 Connection Established \r\n")} else {server.write (b[:n])} // client) io.Copy(client, server)Copy the code

There is a separate response to the CONNECT method, where the client says it wants to establish a connection, and the proxy responds that it has established a connection before it can request access like HTTP.

The complete code

At this point, our proxy server development is complete, the following is the complete source code:

package mainimport ( "bytes" "fmt" "io" "log" "net" "net/url" "strings")func main() { log.SetFlags(log.LstdFlags|log.Lshortfile) l, err := net.Listen("tcp", ":8081") if err ! = nil { log.Panic(err) } for { client, err := l.Accept() if err ! = nil { log.Panic(err) } go handleClientRequest(client) } }func handleClientRequest(client net.Conn) { if client == nil { return } defer client.Close() var b [1024]byte n, err := client.Read(b[:]) if err ! = nil { log.Println(err) return } var method, host, address string fmt.Sscanf(string(b[:bytes.IndexByte(b[:], '\n')]), "%s%s", &method, &host) hostPortURL, err := url.Parse(host) if err ! Log.println (err) return} if hostPorturl.opaque == "443" {// HTTPS access address = hostPorturl.scheme + ":443"} Else {// HTTP access if strings.Index(hostPorturl.host, ":") == -1 {// Host without port, Address = hostPorturl.host + ":80"} else {address = hostPorturl.host}} Server, err := net.Dial(" TCP ", address) if err! = nil { log.Println(err) return } if method == "CONNECT" { fmt.Fprint(client, "HTTP/1.1 200 Connection Established \r\n")} else {server.write (b[:n])} // client) io.Copy(client, server) }Copy the code

Compile the source code, put it on your own computer, and test it out.