preface

9, Netease Netease

I. Page analysis


1. Press F12 to enter two words of debugging input crawler

2. Parameter parsing


  1. Salt: 13-bit timestamp + a random number
  2. Sign: encryption parameter
  3. LTS: 13-bit timestamp
  4. Bv: encryption parameter of the browser version


Second, parameter cracking


1. Search for the encryption location of parameters

1.1 Old rules: Search for sign directly and click on it directly



1.2 Search for sign after entering and find the encrypted location



1.3 breakpoint debugging, you can obtain the encryption mode

  1. Salt: 13-bit timestamp + a random number
  2. Sign: MD5 (‘fanyideskweb’+ translation parameter +’)zqSxf-=B#7m’)
  3. LTS: 13-bit timestamp
  4. Bv: MD5 (browser version)


2. Parameter cracking test

2.1 Can be seen to be successful ~


Three, code,

import requests
import execjs

cookies = {
    'OUTFOX_SEARCH_USER_ID': '1 @1.1.1.1'.# this must be added, create a random format: [email protected]
}

headers = {
    'Connection': 'keep-alive'.'Accept': 'application/json, text/javascript, */*; Q = 0.01 '.'X-Requested-With': 'XMLHttpRequest'.'User-Agent': 'the Mozilla / 5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36'.'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'.'Origin': 'http://fanyi.youdao.com'.'Referer': 'http://fanyi.youdao.com/'.'Accept-Language': 'zh-CN,zh; Q = 0.9 ',}with open('code.js', encoding='utf8') as f:
    js = f.read()
js_func = execjs.compile(js) # JS reads and compiles

translation_key = 'crawlers'
encode_params = js_func.call('get_params',translation_key) Get encryption parameters
print(encode_params,'\n')

data = {
  'i': translation_key,
  'from': 'AUTO'.'to': 'AUTO'.'smartresult': 'dict'.'client': 'fanyideskweb'.'salt': encode_params['salt'].'sign': encode_params['sign'].'lts': encode_params['ts'].'bv': encode_params['bv'].'doctype': 'json'.'version': '2.1'.'keyfrom': 'fanyi.web'.'action': 'FY_BY_REALTlME'
}

response = requests.post('http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule', headers=headers,cookies=cookies, data=data, verify=False)

print(response.text)
Copy the code
var CryptoJS = require('crypto-js'); // nodeJS encryption package is used here

function get_params(data) {
    var useragent = "5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36";
    var bv = CryptoJS.MD5(useragent).toString()
        , ts = "" + (new Date).getTime()
        , salt = ts + parseInt(10 * Math.random(), 10);
    var sign = CryptoJS.MD5('fanyideskweb'+data+salt+']BjuETDhU)zqSxf-=B#7m').toString();
    return {bv: bv, ts: ts, salt: salt, sign: sign}
}

console.log(get_params('crawlers'));
Copy the code