Crawler reverse basis, understanding SM1-SM9, ZUC state secret algorithm

Pay attention to wechat public number: K brother reptile, QQ exchange group: 808574309, continue to share crawler advanced, JS/ Android reverse technology and other goods!

【 01X00 】 Introduction

Countries of the state password administration decided that domestic encryption algorithm, JS crawler engineers are doing the reverse, will meet all kinds of encryption algorithms, including RSA, AES and SHA algorithm is the most common, these algorithms are all abroad, brother in the K in the previous articles have introduced: the crawler common encryption to decrypt the crawler knowledge 】 【 algorithm”

In fact, since 2010, China’s National Cryptography Administration has begun to release a series of domestic encryption algorithms, including SM1, SM2, SM3, SM4, SM7, SM9, ZUC (Zuchong encryption algorithm), SM stands for business secret, that is, commercial password, Refers to commercial cryptographic technology that does not involve state secrets. The algorithms of SM1 and SM7 are not public. The other algorithms have become ISO/IEC international standards.

Among these domestic encryption algorithms, SM2, SM3 and SM4 encryption algorithms are relatively common, and they may also be encountered when crawling some GOV websites. Therefore, as a crawler engineer, it is necessary to know about these algorithms. As shown in the figure below, a gov website uses SM2 and SM4 encryption algorithms:

【 02X00 】 Overview of algorithm

【 03X00 】 Algorithm in detail

[03X01] SM1 block encryption algorithm

SM1 is a packet encryption algorithm with symmetric encryption. The packet length and key length are both 128 bits. Therefore, when encrypting and decrypting messages, if the message length is too long, you need to group them. Performance security strength and related hardware and software implementation algorithm and AES, the algorithm does not open, only exists in the form of IP core chip, calls the algorithm, need through the encryption chip interface calls, adopting the algorithm has developed a series of chips, smart IC card, smart encrypted password keys, encryption card, and other security products, It is widely used in various application fields of e-government, e-commerce and national economy (including national government affairs, police affairs and other important fields). Few people generally understand it, and crawler engineers will not encounter this encryption algorithm.

SM2 Elliptic curve public key encryption algorithm

SM2 is elliptic curve (ECC) public key encryption algorithm, asymmetric encryption, SM2 algorithm and RSA algorithm are public key encryption algorithm, SM2 algorithm is a more advanced security algorithm, in our country’s commercial cryptography system is used to replace the RSA algorithm, in many Gov sites to see this kind of encryption algorithm. Chinese scholars began to study elliptic curve cryptography in the 1980s, and many achievements have been made. SM2 elliptic curve public key cryptography algorithm has the following advantages over RSA algorithm:

[03×03] SM3 hash algorithm

SM3 is a password hashing algorithm that uses the password hash function standard to replace MD5, SHA-1, sha-2 and other international algorithms. SM3 is an improved algorithm based on SHA-256. The length of message groups is 512 bits, and the length of digest values is 256 bits. It uses xOR, module, modular addition, shift, and, or, and not operations, consisting of padding, iterative processes, message expansion, and compression functions. In commercial cryptography, SM3 is mainly used for digital signature and authentication, message authentication code generation and authentication, and random number generation. According to the National Cryptography Administration, it is more secure and efficient than MD5 and SHA-1 algorithms, and comparable to SHA-256.

[03×04] SM4 packet encryption algorithm

SM4 is a wireless LAN standard packet encryption algorithm. Symmetric encryption is used to replace international algorithms such as DES/AES. SM4 and AES have the same key length and packet length, both of which are 128 bits. Fill. The encryption algorithm and the key extension algorithm both adopt the nonlinear iterative structure of 32 rounds. The decryption algorithm and the encryption algorithm have the same structure, but the order of use of the round key is opposite. The decryption round key is the reverse order of the encryption round key.

[03×05] SM7 packet encryption algorithm

SM7 for grouping encryption algorithms, symmetric encryption, the algorithm does not open, including identity class application (contactless IC card, access control card, work permit, entry permit, etc.), ticket application class (large event tickets, tickets for the exhibition, etc.), payment and pass application class (integral be, campus id, enterprise one cartoon, etc.). Crawler engineers rarely encounter such algorithms.

[03X06] SM9 Identifies the encryption algorithm

SM9 is an identity-based Cryptography algorithm, asymmetric encryption. Identity encryption takes the user’s Identity (such as wechat, email address, mobile phone number, QQ number, etc.) as a public key, omits the process of exchanging digital certificates and public keys, making the security system easy to deploy and manage. Security guarantee of various emerging applications applicable to Internet applications, such as password service based on cloud technology, email security, intelligent terminal protection, Internet of things security, cloud storage security and so on. These security applications can use mobile phone numbers or email addresses as public keys to encrypt data, identity authentication, call encryption, and channel encryption. In commercial cryptography, SM9 is mainly used for user identity authentication. According to xinhuanet.com, THE encryption strength of SM9 is equal to that of RSA encryption algorithm with 3,072 bits of key.

【03×07】 The algorithm of ZUC

ZUC is a stream cryptography algorithm with symmetric encryption. The confidentiality algorithm can be applied to encryption and decryption in 3GPP LTE communication. The algorithm includes three parts: ZUC, confidentiality algorithm (128-EEA3) and integrity algorithm (128-EIA3). It has been recommended by the international organization 3GPP as the candidate algorithm of the third set of international encryption and integrity standards for 4G wireless communication.

[04X00] programming language implementation

[04×01] Python implementation

There are no official libraries in Python to implement the national secret algorithm, and only two of them are listed here. It should be noted that SM1 and SM7 algorithms are not public, and most libraries only implement SM2, SM3, and SM4 encryption algorithms.

• Snowland-smx-python:gitee.com/snowlandltd…
• Gmssl:github.com/duanhongyi/…
• Gmssl-python:github.com/gongxian-di…

Gmssl-python is an improved version of GMSSL. Gmssl-python supports SM9 algorithm. However, as of the writing of this article, GMSSL-Python has not released PYPI, nor PR GMSSL. GMSSL installed using PIP install GMSSL does not support SM9 algorithm. To use the SM9 algorithm, download the gmSSL-Python source code and install it manually.

Take THE SM2 algorithm of GMSSL as an example, the implementation is as follows (for other algorithms and detailed usage, please refer to its official documentation) :

SM2 Encryption (encrypt) and Decrypt:

from gmssl import sm2


Hexadecimal public and private keys
private_key = '00B9AB0B828FF68872F21A837FC303668428DEA11DCD1B24429D0C99E24EED83D5'
public_key = 'B9C9A6E04E9C91F7BA880429273747D7EF5DDEB0BB2FF6317EB00BEF331A83081A6994B8993F3F5D6EADDDB81872266C87C018FB4162F5AF347B483 E24620207'
sm2_crypt = sm2.CryptSM2(public_key=public_key, private_key=private_key)

The data to be encrypted and the data to be encrypted are bytes
data = b"this is the data to be encrypted"
enc_data = sm2_crypt.encrypt(data)
dec_data = sm2_crypt.decrypt(enc_data)

print('enc_data: ', enc_data.hex())
print('dec_data: ', dec_data)

# enc_data: 3cb96dd2e0b6c24df8e22a5da3951d061a6ee6ce99f46a446426feca83e501073288b1553ca8d91fad79054e26696a27c982492466dafb5ed06a573f b09947f2aed8dfae243b095ab88115c584bb6f0814efe2f338a00de42b244c99698e81c7913c1d82b7609557677a36681dd10b646229350ad0261b51 ca5ed6030d660947

# dec_data: b'this is the data to be encrypted'
Copy the code

SM2 signature and verify:

from gmssl import sm2, func


Hexadecimal public and private keys
private_key = '00B9AB0B828FF68872F21A837FC303668428DEA11DCD1B24429D0C99E24EED83D5'
public_key = 'B9C9A6E04E9C91F7BA880429273747D7EF5DDEB0BB2FF6317EB00BEF331A83081A6994B8993F3F5D6EADDDB81872266C87C018FB4162F5AF347B483 E24620207'
sm2_crypt = sm2.CryptSM2(public_key=public_key, private_key=private_key)

The data to be signed is bytes
data = b"this is the data to be signed"
random_hex_str = func.random_hex(sm2_crypt.para_len)

# hexadecimal
sign = sm2_crypt.sign(data, random_hex_str)
verify = sm2_crypt.verify(sign, data)

print('sign: ', sign)
print('verify: ', verify)

# sign:  45cfe5306b1a87cf5d0034ef6712babdd1d98547e75bcf89a17f3bcb617150a3f111ab05597601bab8c41e2b980754b74ebe9a169a59db37d549569910ae273a

# verify: True
Copy the code

【04×02】JavaScript language

Sm-crypto is recommended here and currently supports SM2, SM3 and SM4. It should be noted that SM2 asymmetric encryption results are composed of C1, C2 and C3, among which C1 is the elliptic curve point calculated to generate random numbers. C2 is ciphertext data, C3 is the summary value of SM3. The results of the original national secret standard are stored in the order of C1C2C3, while the new standard is stored in the order of C1C3C2. Sm-crypto supports setting cipherMode, that is, the order of C1C2C3.

Sm-crypto:www.npmjs.com/package/sm-…

Take SM2 algorithm as an example, the implementation is as follows (for other algorithms and detailed usage, please refer to its official documentation) :

SM2 Encryption (encrypt) and Decrypt:

const sm2 = require('sm-crypto').sm2

// 1-c1C3C2, 0-c1C2C3. The default value is 1
const cipherMode = 1

// Get the key pair
let keypair = sm2.generateKeyPairHex()
let publicKey = keypair.publicKey   / / the public key
let privateKey = keypair.privateKey / / the private key

let msgString = "this is the data to be encrypted"
let encryptData = sm2.doEncrypt(msgString, publicKey, cipherMode)    // Encrypt the result
let decryptData = sm2.doDecrypt(encryptData, privateKey, cipherMode) // Decrypt the result

console.log("encryptData: ", encryptData)
console.log("decryptData: ", decryptData)

// encryptData: ddf261103fae06d0efe20ea0fe0d82bcc170e8efd8eeae24e9559b3835993f0ed2acb8ba6782fc21941ee74ca453d77664a5cb7dbb91517e6a3b0c27 db7ce587ae7af54f8df48d7fa822b7062e2af66c112aa57de94d12ba28e5ba96bf4439d299b41da4a5282d054696adc64156d248049d1eb1d0af28d7 6b542fe8a95d427e

// decryptData: this is the data to be encrypted
Copy the code

SM2 signature and verify:

const sm2 = require('sm-crypto').sm2

// Get the key pair
let keypair = sm2.generateKeyPairHex()
let publicKey = keypair.publicKey   / / the public key
let privateKey = keypair.privateKey / / the private key

// Pure signature + generates elliptic curve points
let msgString = "this is the data to be signed"
let sigValueHex = sm2.doSignature(msgString, privateKey)                    / / signature
let verifyResult = sm2.doVerifySignature(msgString, sigValueHex, publicKey) // Test result

console.log("sigValueHex: ", sigValueHex)
console.log("verifyResult: ", verifyResult)

// sigValueHex:  924cbb9f2b5adb554ef77129ff1e3a00b2da42017ad3ec2f806d824a77646987ba8c8c4fb94576c38bc11ae69cc98ebbb40b5d47715171ec7dcea913dfc6ccc1

// verifyResult: true
Copy the code

【04×03】 Other language implementations and references

• Java language implementation:
  • Github.com/bcgit/bc-cs…
  • Github.com/xjfuuu/SM2_…
• Go language implementation: github.com/tjfoc/gmsm
• Open source state secret algorithm toolbox: gmSSL.org/
• The secret algorithm source code download: www.scctc.org.cn/templates/D…
• National Cryptography Administration: www.sca.gov.cn/
• Ciphers Standards Committee: www.gmbz.org.cn/

[05X00] Attached: GM/T password industry standard

• GM/T 0001.1-2012: Zu Chongzhi sequence cryptography algorithm: Part 1: Algorithm description
• GM/T 0001.2-2012: Sequence cryptography algorithm for zu Chong: Part 2: Confidentiality algorithm based on Zu Chong Zhi algorithm
• GM/T 0001.3-2012: Sequence cryptography algorithm for Zu Chong: Part 3: Integrity algorithm based on Zu Chong Zhi algorithm
• GM/T 0003.1-2012: SM2 Elliptic curve public key cryptography algorithms – Part 1: General
• GM/T 0003.2-2012: SM2 elliptic curve public key cryptography algorithm – Part 2: digital signature algorithm
• GM/T 0003.3-2012: SM2 Elliptic curve public key cryptography algorithm – Part 3: key exchange protocol
• GM/T 0003.4-2012: SM2 Elliptic curve public key cryptography algorithm — Part 4: Public key cryptography algorithm
• GM/T 0003.5-2012: SM2 Elliptic curve public key cryptography algorithms – Part 5: Parameter definitions
• GM/T 0004-2012: SM3 password hashing algorithm
• GM/T 0002-2012: SM4 block password algorithm
• GM/T 0044.1-2016: SM9 Identification cryptographic algorithms – Part 1: General
• GM/T 0044.2-2016: SM9 Identification cryptographic algorithm – Part 2: Digital signature algorithm
• GM/T 0044.3-2016: SM9 Identification cryptographic algorithm – Part 3: Key exchange protocol
• GM/T 0044.4-2016: SM9 Identification cryptographic algorithms – Part 4: Key encapsulation mechanisms and public key encryption algorithms
• GM/T 0044.5-2016: SM9 Identification cryptography algorithms – Part 5: Parameter definitions