Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

Spring’s Resttemplates are familiar to all of us. If you are not familiar with them, you can learn more about Resttemplates in Springboot: Making HTTP Requests in a More elegant way. The RestTemplate is very simple and convenient to use, but there are a few things to note about URLEncode

The requested URL (string type) is automatically URLEncode

Test code:

RestTemplate restTemplate = new RestTemplate(); RestTemplate. GetForEntity (" https://www.baidu.com?r= code ", String class);Copy the code

Console log:

Source code analysis:

URLEncode the URL string with the following code

URI expanded = this.getUriTemplateHandler().expand(url, uriVariables);
Copy the code

Specific process:

  1. Via the url generated DefaultUriBuilderFactory DefaultUriBuilder object
new DefaultUriBuilderFactory.DefaultUriBuilder(uriTemplate)
Copy the code

  1. Call DefaultUriBuilderFactory. The build method of DefaultUriBuilder URI

    2.1 through the implementation class UriComponents HierarchicalUriComponents has parsing HierarchicalUriComponents objects generated in the form of stratification

    this.expandInternal(new UriComponents.VarArgsTemplateVariables(uriVariableValues));
Copy the code
    protected HierarchicalUriComponents expandInternal(UriTemplateVariables uriVariables) { Assert.state(! this.encodeState.equals(HierarchicalUriComponents.EncodeState.FULLY_ENCODED), "URI components already encoded, and could not possibly contain '{' or '}'."); String schemeTo = expandUriComponent(this.getScheme(), uriVariables, this.variableEncoder); String userInfoTo = expandUriComponent(this.userInfo, uriVariables, this.variableEncoder); String hostTo = expandUriComponent(this.host, uriVariables, this.variableEncoder); String portTo = expandUriComponent(this.port, uriVariables, this.variableEncoder); HierarchicalUriComponents.PathComponent pathTo = this.path.expand(uriVariables, this.variableEncoder); MultiValueMap<String, String> queryParamsTo = this.expandQueryParams(uriVariables); String fragmentTo = expandUriComponent(this.getFragment(), uriVariables, this.variableEncoder); return new HierarchicalUriComponents(schemeTo, fragmentTo, userInfoTo, hostTo, portTo, pathTo, queryParamsTo, this.encodeState, this.variableEncoder); }Copy the code

    2.2 Encode encapsulated Uri components to generate Uri objects

    this.createUri(uric);
Copy the code
    private URI createUri(UriComponents uric) {
    if (DefaultUriBuilderFactory.this.encodingMode.equals(DefaultUriBuilderFactory.EncodingMode.URI_COMPONENT)) {
        uric = uric.encode();
    }
    return URI.create(uric.toString());
}

Copy the code

    2.3 Encode Uri parts

    public HierarchicalUriComponents encode(Charset charset) { if (this.encodeState.isEncoded()) { return this; } else { String scheme = this.getScheme(); String fragment = this.getFragment(); String schemeTo = scheme ! = null ? encodeUriComponent(scheme, charset, HierarchicalUriComponents.Type.SCHEME) : null; String fragmentTo = fragment ! = null ? encodeUriComponent(fragment, charset, HierarchicalUriComponents.Type.FRAGMENT) : null; String userInfoTo = this.userInfo ! = null ? encodeUriComponent(this.userInfo, charset, HierarchicalUriComponents.Type.USER_INFO) : null; String hostTo = this.host ! = null ? encodeUriComponent(this.host, charset, this.getHostType()) : null; BiFunction<String, HierarchicalUriComponents.Type, String> encoder = (s, type) -> { return encodeUriComponent(s, charset, type); }; HierarchicalUriComponents.PathComponent pathTo = this.path.encode(encoder); MultiValueMap<String, String> queryParamsTo = this.encodeQueryParams(encoder); return new HierarchicalUriComponents(schemeTo, fragmentTo, userInfoTo, hostTo, this.port, pathTo, queryParamsTo, HierarchicalUriComponents.EncodeState.FULLY_ENCODED, (UnaryOperator)null); }}Copy the code

    Specific encoding (core logic: determine whether a character needs encoding and URLEncode if necessary) :

        static String encodeUriComponent(String source, Charset charset, HierarchicalUriComponents.Type type) {
        if (!StringUtils.hasLength(source)) {
            return source;
        } else {
            Assert.notNull(charset, "Charset must not be null");
            Assert.notNull(type, "Type must not be null");
            byte[] bytes = source.getBytes(charset);
            boolean original = true;
            byte[] var5 = bytes;
            int var6 = bytes.length;

            int var7;
            for(var7 = 0; var7 < var6; ++var7) {
                byte b = var5[var7];

                if (!type.isAllowed(b)) {
                    original = false;
                    break;
                }
            }

            if (original) {
                return source;
            } else {
                ByteArrayOutputStream baos = new ByteArrayOutputStream(bytes.length);
                byte[] var13 = bytes;
                var7 = bytes.length;

                for(int var14 = 0; var14 < var7; ++var14) {
                    byte b = var13[var14];
                    if (type.isAllowed(b)) {
                        baos.write(b);
                    } else {
                        baos.write(37);
                        char hex1 = Character.toUpperCase(Character.forDigit(b >> 4 & 15, 16));
                        char hex2 = Character.toUpperCase(Character.forDigit(b & 15, 16));
                        baos.write(hex1);
                        baos.write(hex2);
                    }
                }

                return StreamUtils.copyToString(baos, charset);
            }
        }
}
Copy the code

Encode and java.net.URLEncode are not completely consistent

Spring’s RestTemplate to encode a url, but its encode and JDK own java.net.URLEncode.encode method is not completely consistent, the difference is: whether characters need to encode different logic.

Example:

RestTemplate restTemplate = new RestTemplate(); RestTemplate. GetForEntity (" https://www.baidu.com?r=, yards, "String. Class); System.out.println(URLEncoder. Encode (" encode ", "utF-8 "));Copy the code

Console log:

16:12:09. 548. [the main] the DEBUG org. Springframework. Web. Client. The RestTemplate - HTTP GET https://www.baidu.com?r=%E7%BC%96, A0 E7% % % 81 16:12:09. [the main] 559 DEBUG org. Springframework. Web. Client. RestTemplate - Accept=[text/plain, application/json, application/*+json, * / *] 16:12:09. 745. [the main] the DEBUG org. Springframework. Web. Client. The RestTemplate - 200 OK Response 16:12:09, 746 [main] the DEBUG org.springframework.web.client.RestTemplate - Reading to [java.lang.String] as "text/html" %E7%BC%96%2C%E7%A0%81Copy the code

For the “code” string

Spring encode results: %E7%BC%96,%E7%A0%81

java.net.URLEncode.encode：%E7%BC%96%2C%E7%A0%81

Difference: Spring does not encode; URLEncode will encode (encoding result %2C)

Characters not encoded by Spring (corresponding to ASCII code) :

/ / reference: org.springframework.web.util.HierarchicalUriComponents.Type#QUERY_PARAM c >= 97 && c <= 122 || c >= 65 && c <= 90 || 33 == c || 36 == c || 39 == c || 40 == c || 41 == c || 42 == c || 43 == c || 44 == c || 59 == c || 58 == c || 64 == cCopy the code

URLEncode Unencoded character (corresponding to ASCII code) :

/ / reference/Library/Java/JavaVirtualMachines/Zulu - 8. The JDK/Contents/Home/SRC. Zip! /java/net/URLEncoder.java:87 for (i = 'a'; i <= 'z'; i++) { dontNeedEncoding.set(i); } for (i = 'A'; i <= 'Z'; i++) { dontNeedEncoding.set(i); } for (i = '0'; i <= '9'; i++) { dontNeedEncoding.set(i); } dontNeedEncoding.set(' '); /* encoding a space to a + is done * in the encode() method */ dontNeedEncoding.set('-'); dontNeedEncoding.set('_'); dontNeedEncoding.set('.'); dontNeedEncoding.set('*');Copy the code

Spring Encode is not exactly the same as java.net.URLEncode, and most usage scenarios are unaffected because URLDecode can be decoded even without transcoding. But when it comes to signature calculations, pay special attention! I recently stepped into a pit like this in a project:

For Baidu open platform, the verification requires URLEncode encoding of parameters and then encryption to generate signatures. If a request is initiated using RestTemplate with a comma character, the check fails. Because the URLEncode parameter on the URL is inconsistent with the URLEncode result when the signature was generated

Execute methodString urlandURI urlThe difference between

Source:

@Nullable public <T> T execute(String url, HttpMethod method, @Nullable RequestCallback requestCallback, @Nullable ResponseExtractor<T> responseExtractor, Map<String, ? > uriVariables) throws RestClientException { URI expanded = this.getUriTemplateHandler().expand(url, uriVariables); return this.doExecute(expanded, method, requestCallback, responseExtractor); } @Nullable public <T> T execute(URI url, HttpMethod method, @Nullable RequestCallback requestCallback, @Nullable ResponseExtractor<T> responseExtractor) throws RestClientException { return this.doExecute(url, method, requestCallback, responseExtractor); }Copy the code

It is easy to know from the source code that if the argument is String URL, it is to handle the URL, such as URLEncode etc.