Go (gin) signature step on the pit

preface

Recently, when accessing the third-party interface, you need to verify the signature in the parameter. The signature is SHA256withRSA (RSA2) to confirm whether the data has been modified. The specific principle of SHA256withRSA is not explained here. This paper mainly records some pits stepped on during go(GIN framework) visa inspection, and summarizes and records them.

Ps: Some of the following code does not do error handling and is not desirable in real development.

SHA256withRSA

Unsigned string

Interface parameters are passed in as x-www-form-urlencoded, as follows

utc_timestamp:1624864579690 sign:LPyc1kQNle9fTNfPi7zDz77eZFG0XD0YBXsRQNw/cCq00YE2dISzZIizi5S30ssHfVS2uuQsOyYYZoI8BgT1VR3vcf3CdOY8rkPPdqhBgcEJyKNRvQ3 z+3VnM33gP84J5Ntg/LS8ZAlGpGjL9xTWtKVUbHZk0oy1qJwt3Da+wqchk5oh/cYeQnTyyUheQBf2WwPeNYCoauUS6R3KCtF3X8d2qUjx2ZEMkAMQhqGG9Dw apWdTdoStjDZt+/Uz2wNT/4ctTa0iTvKPh5Zn1fBhBEKiflXlC32tRjS5hC2RfXR/1JR/AF+u937THwZmWv4xDPAQwRNcNwIH+a6mafygKg== sign_type:RSA2 app_id:20210701 content:{"page":1,"size":20}Copy the code

Assemble string to be signed:

Get all parameters, remove sign and sign_type parameters
The filtered parameters are sorted by the ASCII key increment of the first character (alphabetically ascending), by the ASCII key increment of the second character if the same character is encountered, and so on
The sorted parameters and their corresponding values are combined in the format of parameter = Parameter value, and these parameters are joined with & characters. In this case, the string to be signed is generated

App_id =20210701&content={” Page “:1,”size”:20}& UTC_TIMESTAMP =1624864579690

Signature and verification

Although it is used as a signature verification party, in order to facilitate the test, the signature method is also implemented. Public key and private key, private key signature and public key signature are prepared first.

Implementing signature functions

func RsaSignWithSha256(data []byte, keyBytes []byte) ([]byte, error) {
	h := sha256.New()
	h.Write(data)
	hashed := h.Sum(nil)
	block, _ := pem.Decode(keyBytes)
	if block == nil {
		return nil, errors.New("private key error")
	}
	privateKey, err := x509.ParsePKCS8PrivateKey(block.Bytes)
	iferr ! =nil {
		return nil, err
	}

	signature, err := rsa.SignPKCS1v15(rand.Reader, privateKey.(*rsa.PrivateKey), crypto.SHA256, hashed)
	iferr ! =nil {
		return nil, err

	}

	return signature, nil
}
Copy the code

The result is a byte slice, but the argument is passed as a string, so you need to slice the byte into a string in two forms:

hex hex.EncodeToString
base64 base64.StdEncoding.EncodeToString

After encoding the data, there will be significant differences

// base64 AezhDSynfsTMrU517zHK12e2SzczNczm+yRht+Dr+I0K7VE+TLeUbpB1SiMbxLIdT2SsunIm0h5vaeHAyf9QwAFvjlcPG6JhJBOo58AtXx2moVVuu2pAEtO/ tJw61VKbT4j5nAIiC1Ac2i1+u5BdbYoAV6Fc+HtfAJBS1iWinwQ= // hex 01ece10d2ca77ec4ccad4e75ef31cad767b64b373335cce6fb2461b7e0ebf88d0aed513e4cb7946e90754a231bc4b21d4f64acba7226d21e6f69e1c0 c9ff50c0016f8e570f1ba2612413a8e7c02d5f1da6a1556ebb6a4012d3bfb49c3ad5529b4f88f99c02220b501cda2d7ebb905d6d8a0057a15cf87b5f 009052d625a29f04Copy the code

No matter which encoding is used, the signature must be decoded and converted into byte slices before verification, otherwise the verification will not pass. The following is the verification function, using HEX

func RsaVerySignWithSha256(data, signData, keyBytes []byte) bool {
	block, _ := pem.Decode(keyBytes)
	if block == nil {
		panic(errors.New("public key error"))
	}
	pubKey, err := x509.ParsePKIXPublicKey(block.Bytes)
	iferr ! =nil {
		panic(err)
	}

	hashed := sha256.Sum256(data)

        // Note that the signature string is decoded here
	/ / sig, _ : = base64 StdEncoding. DecodeString (string (signData) base64 decoding
	sig, _ := hex.DecodeString(string(signData)) / / hex decoding

	err = rsa.VerifyPKCS1v15(pubKey.(*rsa.PublicKey), crypto.SHA256, hashed[:], sig)
	iferr ! =nil {
		panic(err)
	}
	return true
}
Copy the code

Finally the whole process of signing and checking is completed

func main (a){
  s := `app_id=20210701&content={"page":1,"size":20}&utc_timestamp=1624864579690`
  sign, _ := RsaSignWithSha256([]byte(s), prvKey)
  sigs := hex.EncodeToString(sign)
  fmt.Println(RsaVerySignWithSha256([]byte(s), []byte(sigs), pubKey)) // true
}
Copy the code

Now that the verification process is completed, it is the next step to apply it in the project. Gin framework is used here, and in the following part, a lot of pit Orz is stepped

Go on pit (Gin)

From the parameter to the pending string

In gin, I always use ShouldBind to bind the parameters to the structure, and this time I use it naturally. Once you’ve bound the parameters to a structure, and you want to spell the signature string, you need to iterate through the structure, and to iterate through the structure, you need to do reflection

func GetPendingSign(p interface{}) []byte {
	var typeInfo = reflect.TypeOf(p)
	var valInfo = reflect.ValueOf(p)

	num := typeInfo.NumField()
	var keys = make([]string.0, num)
	var field = make([]string.0, num)
	for i := 0; i < num; i++ {
		key := typeInfo.Field(i).Tag.Get("form") // The struct form tag is the key of the string to be signed
		ifkey ! ="sign"&& key ! ="sign_type" {
			keys = append(keys, key)
		  field[key] = typeInfo.Field(i).Name // Create a form tag that matches the attribute name
		}
	}

	sort.Strings(keys)
  
	s := ""
	for i, k := range keys {
		temp := valInfo.FieldByName(field[i]).Interface() // Get the value from the corresponding value above
		if k == "content" {
                       // Since the content is deserialized, it is reserialized to JSON to concatenate strings
			b, _ := json.Marshal(temp)
			s = fmt.Sprintf("%s%s=%s&", s, k, string(b))
		} else {
			s = fmt.Sprintf("%s%s=%v&", s, k, temp)
		}
	}
	s = s[:len(s)- 1] // A character string to be signed
	return []byte(s)
}
Copy the code

Several sections of the above method are commented, which are also critical. We then test it with Postman, passing in the initial argument, and get the same checkable string as expected.

  r := gin.Default()
	r.POST("/test".func(c *gin.Context) {
		var body Body
		c.ShouldBind(&body)
		fmt.Println(string(GetPendingSign(body)))
	})
	r.Run(": 8081")
Copy the code

But is this the end? No! This way, there is a big problem! Content :{“page”:1,”size”:20} “content:{“size”:20,”page”:1}” Yes! The problem with this approach is that the size and page are transposed.

Structure /map and JSON

Why did the inspection fail? The first idea is are the strings to be signed consistent? Content ={“page”:1,”size”:20} instead of content:{“size”:20,”page”:1} You changed places? Let’s look at the definition of a structure

type Content struct {
	Page int `json:"page" form:"page"`
	Size int `json:"size" form:"size"`
}
Copy the code

The serialization sequence of the key values is the same as the sequence in which the attributes of the structure were defined, rather than the original JSON

Now that we’re talking about structures, let’s look at maps

	s := `{"size":20,"page":1}`
	m := make(map[string]int)
	json.Unmarshal([]byte(s), &m)
	b, _ := json.Marshal(&m)
	fmt.Println(s)
	fmt.Println(string(b)) // {"page":1,"size":20}
Copy the code

The order of keys is also adjusted. What rules does map use to adjust the order of keys?

See this line at line 793 in encoding/json/encode.go of the source code

sort.Slice(sv, func(i, j int) bool { return sv[i].s < sv[j].s }) 
Copy the code

That is, map to JSON is in order, and keys are arranged in ascending ASCII order.

Well, since both methods change the order of the keys, binding the structure first and then iterating through the concatenation is not an option.

The solution

Since you can’t deserialize first, you have to do something else. Instead of getting the argument ShouldBind, use ioutil.ReadAll and print it out

utc_timestamp=1624864579690&sign=LPyc1kQNle9fTNfPi7zDz77eZFG0XD0YBXsRQNw%2FcCq00YE2dISzZIizi5S30ssHfVS2uuQsOyYYZoI8BgT1V R3vcf3CdOY8rkPPdqhBgcEJyKNRvQ3z%2B3VnM33gP84J5Ntg%2FLS8ZAlGpGjL9xTWtKVUbHZk0oy1qJwt3Da%2Bwqchk5oh%2FcYeQnTyyUheQBf2WwPeN YCoauUS6R3KCtF3X8d2qUjx2ZEMkAMQhqGG9DwapWdTdoStjDZt%2B%2FUz2wNT%2F4ctTa0iTvKPh5Zn1fBhBEKiflXlC32tRjS5hC2RfXR%2F1JR%2FAF% 2Bu937THwZmWv4xDPAQwRNcNwIH%2Ba6mafygKg%3D%3D&sign_type=RSA2&app_id=20210701&content=%7B%22size%22%3A20%2C%22page%22%3A1 %7DCopy the code

The parameter form of X-www-form-urlencoded is similar to the parameter form of query, both of which are concatenated with & and =. Since they are strings, they are cut manually, then put together into a map, and finally pass through the map and concatenated into a string to be signed.

	bodyArray := strings.Split(string(body), "&") //1. Press &cut first
	data := make(map[string]string)
	for _, v := range bodyArray {
		// 2, assemble map according to = split
		vs := strings.Split(v, "=")
		if len(vs) == 2 {
			value, err := url.QueryUnescape(vs[1]) // From the above printed characters, it can be seen that the urlescape, so Unescape
			iferr ! =nil {
				c.Abort()
				return
			}
			data[vs[0]] = value
		}
	}
Copy the code

In the form above, you get a map, and the content value, because it’s just a string, is not deserialized and then serialized. So there is no inconsistency of key order. Then you simply iterate over the map and assemble the checkable strings as required.

Finally, these steps are encapsulated into a middleware for use, and the verification function is completed.

conclusion

Take a look at the final knowledge:

The signature has base64 and HEX encoding mode, and the corresponding decoding should be performed when checking the signature
Use reflection to traverse the structure
When a structure is serialized to JSON, the JSON key is reordered in order of the structure’s attributes
When a map is serialized as JSON, the JSON keys are in ascending ASCII order
X-www-form-urlencoded parameters, concatenated with & and =, and will be urlescape, unescape for processing
One last tip, middlewareioutil.ReadAllAfter reading the body, put the body backc.Request.Body = ioutil.NopCloser(bytes.NewBuffer(body))Otherwise, the following route will not read the body

The road is long

Thanks!

preface

SHA256withRSA

Unsigned string

Signature and verification

Go on pit (Gin)

From the parameter to the pending string

Structure /map and JSON

The solution

conclusion

Related Posts

Python project mirror slimming practice

Java Collection (1) — Collection

Java Multiple file reading methods