[Go] Obtain the real IP address of the user

The original link: blog.thinkeridea.com/201903/go/g…

There are many links between the user request and the server that provides the service, which makes it very difficult for the service to obtain the real IP of the user. Most frameworks and tool libraries encapsulate various methods to obtain the real IP of the user. Exnet package also encapsulates various IP-related operations, including the method to obtain the client IP. The more practical methods are as follows:

• func ClientIP(r *http.Request) stringClientIP does its best to implement an algorithm for obtaining ClientIP. Resolves x-real-IP and X-Forwarded-For so that the reverse proxy (NGINx or HaProxy) can work properly.
• func ClientPublicIP(r *http.Request) stringClientPublicIP tries its best to implement the algorithm for obtaining the IP address of the client’s public network. Resolves x-real-IP and X-Forwarded-For so that the reverse proxy (NGINx or HaProxy) can work properly.
• func HasLocalIP(ip net.IP) boolHasLocalIP Checks whether an IP address is an Intranet address
• func HasLocalIPddr(ip string) boolHasLocalIPddr Checks whether the IP address string is an Intranet address
• func RemoteIP(r *http.Request) stringRemoteIP Gets the IP address from RemoteAddr, just a quick resolution method.

Obtain the real IP address of the user

The ClientIP method is similar to the ClientPublicIP method, except that one obtains the ClientIP address according to the HTTP protocol and the other obtains the public IP address according to the protocol.

In an environment with complex network and service architecture and business logic, it is not always possible to obtain the real IP according to the HTTP protocol. In our business, user traffic is forwarded by three parties at multiple levels (all are HTTP clients implemented by the three parties themselves), which inevitably causes some errors. In this case, it becomes more and more difficult for later services to obtain the real IP of users, and you don’t even know whether the IP you get is real or not.

However, most of our clients’ traffic forwarded by three parties, even excluding the test, are public network users. The combination of ClientPublicIP and ClientIP can always better obtain the real IP address of the user.

// var r *http.Request
ip := exnet.ClientPublicIP(r)
if ip == ""{
  ip = exnet.ClientIP(r)
}
Copy the code

The above method can always effectively obtain the real IP address of the user, the following analysis of the concrete implementation of the next two methods.

// ClientIP tries its best to implement the algorithm for obtaining ClientIP.
Http_forwarded_for // Resolve the x-real-IP to X-Forwarded-For so that the reverse proxy (Nginx or HaProxy) works properly.
func ClientIP(r *http.Request) string {
	xForwardedFor := r.Header.Get("X-Forwarded-For")
	ip := strings.TrimSpace(strings.Split(xForwardedFor, ",") [0])
	ifip ! ="" {
		return ip
	}

	ip = strings.TrimSpace(r.Header.Get("X-Real-Ip"))
	ifip ! ="" {
		return ip
	}

	if ip, _, err := net.SplitHostPort(strings.TrimSpace(r.RemoteAddr)); err == nil {
		return ip
	}

	return ""
}
Copy the code

ClientIP first reads the first IP address separated by the x-Forwarded-For header. If this address doesn’t exist, it gets it from the X-Real-IP header. If it still doesn’t exist, the traffic is not Forwarded by a reverse proxy. Instead, the client requests the service directly, using the http.request. RemoteAddr field to intercept the IP address without the port number.

This method is simple. It is in the HTTP convention format, where X-Forwarded-For and X-real-IP headers are populated by a reverse proxy, such as Nginx or HaProxy.

// ClientPublicIP tries its best to implement the algorithm for obtaining the client's public network IP address.
Http_forwarded_for // Resolve the x-real-IP to X-Forwarded-For so that the reverse proxy (Nginx or HaProxy) works properly.
func ClientPublicIP(r *http.Request) string {
	var ip string
	for _, ip = range strings.Split(r.Header.Get("X-Forwarded-For"), ",") {
		ip = strings.TrimSpace(ip)
		ifip ! ="" && !HasLocalIPddr(ip) {
			return ip
		}
	}

	ip = strings.TrimSpace(r.Header.Get("X-Real-Ip"))
	ifip ! ="" && !HasLocalIPddr(ip) {
		return ip
	}

	if ip, _, err := net.SplitHostPort(strings.TrimSpace(r.RemoteAddr)); err == nil {
		if! HasLocalIPddr(ip) {return ip
		}
	}

	return ""
}
Copy the code

If the x-Forwarded-For IP address is Forwarded to forwarded-For, you can set it to forwarded-for. If the x-Forwarded-For IP address is Forwarded to Forwarded-For, you can set it to Forwarded-for. If the x-Forwarded-For IP address is forwarded-for, you can set it to Forwarded-for. Second, check whether http.request. RemoteAddr is a public IP address, and return an empty string if no public IP address is found.

This method gives us a chance to obtain the public IP address of the user first, which is more valuable to us.

Check whether the IP pair is an Intranet address

Exnet also provides the function of checking whether an IP address is an Intranet address, which is useful in some cases. For example, some interfaces in the service can be accessed only by the Intranet, that is, only administrators are allowed to access them (for example, dynamically setting the log level and viewing the service Pprof information). We want to hide the back-end service and only expose the user load balancer (reverse proxy) without direct access to our service. These methods are extremely useful. Let’s see how they are implemented.

My service provides the dynamic setting of log levels, so that when the service fails, you can view the debug log immediately to analyze the specific cause. However, this interface is dangerous and should not be exposed to the public network. Therefore, the routing middleware is used to check whether the request is from the public network, and 404 is returned if it is from the public network.

This method considers that the following address segments are Intranet addresses:

10.0. 0. 0/8
169.254. 0. 0/16
172.16. 0. 0/12
172.17. 0. 0/12
172.18. 0. 0/12
172.19. 0. 0/12
172.20. 0. 0/12
172.21. 0. 0/12
172.22. 0. 0/12
172.23. 0. 0/12
172.24. 0. 0/12
172.25. 0. 0/12
172.26. 0. 0/12
172.27. 0. 0/12
172.28. 0. 0/12
172.29. 0. 0/12
172.30. 0. 0/12
172.31. 0. 0/12
192.168. 0. 0/16
Copy the code

HasLocalIPddr Checks whether the IP address string is an Intranet address
func HasLocalIPddr(ip string) bool {
	return HasLocalIP(net.ParseIP(ip))
}

HasLocalIP Checks whether an IP address is an Intranet address
func HasLocalIP(ip net.IP) bool {
	for _, network := range localNetworks {
		if network.Contains(ip) {
			return true}}return ip.IsLoopback()
}
Copy the code

The difference between the two check methods is that the parameter types are inconsistent. During the check, check whether the IP address is contained in the Intranet IP segment one by one. If not, check whether the IP address is a loopback address.

Obtain the reverse proxy IP address

A layer 4 reverse proxy can even provide the user’s real IP address (http.request.RemoteAddr is the user’s IP address, not the reverse proxy’s IP address) while hiding its own IP address. Here are some common methods.

Http.request. RemoteAddr holds the IP address of the last client connected to the service. The simplest and most effective way to obtain the IP address of the reverse proxy is through http.request. RemoteAddr. Exnet provides a shortcut to RemoteIP as follows:

// RemoteIP fetch IP address from RemoteAddr, just a quick resolution method.
func RemoteIP(r *http.Request) string {
	if ip, _, err := net.SplitHostPort(strings.TrimSpace(r.RemoteAddr)); err == nil {
		return ip
	}

	return ""
}
Copy the code

This is a handy scaffolding that simply splits the IP and port of http.request.remoteADDR and returns a valid IP address, but simplifies writing business code.

Transfer:

Author: Qi Yin (thinkeridea)

Links to this article:Blog.thinkeridea.com/201903/go/g…

Copyright Notice: All articles on this blog are used unless otherwise statedCC BY 4.0CNLicense agreement. Reprint please indicate the source!