Project address to find a star
At present, businesses do not sell goods for a year, double 11 to sell goods for a year is a fact that we all know, in general, adjust the price of mosquito legs, better than nothing, but there will be some god price will appear, at this time to buy is to earn
Originally, I wanted to buy a Z370 board U set while double 11 sets of computers, but I didn’t expect that jingdong 8700K has been out of stock. These days, it is available, and the price has risen to 3999, which I can’t bear. After looking at the board U set, it is cheaper, but some board U sets do not support automatic ordering. Gayhub searches to see if there are crawlers that can listen for the delivery of goods and automatically place orders. We have jD-Autobuy Python scripts and Go Python scripts
The HTTP library used this time is AXIos, which supports both client and server. Generally speaking, the syntax is very simple. There is also a superagent library before this, which is similar after a look, but superagent processes more on Response
Since I used Axios in Vue, I want to test the capability of the server side this time. It is still good as usual
Write a request header. After all, it’s a server. There’s no browser to handle user-Agent for you, so go to the browser and ask for the header
Const defaultInfo = {header: {' user-agent ': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36', 'Content-type ': 'text/plain; charset=utf-8', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language': 'zh-CN,zh; Q = 0.8, useful - TW; Q = 0.6, en. Q = 0.4, en - US; Q =0.2', 'Connection': 'keep-alive',},}Copy the codeWhen we get the header, we can disguised as a browser to request two-dimensional code pictures. Jingdong’s scanning image address is https://qr.m.jd.com/show. There is no extra skill, but we can directly use AXIos to make a GET request
async function requestScan() {
const result = await request({
method: 'get',
url: 'https://qr.m.jd.com/show',
headers: defaultInfo.header,
params: {
appid: 133,
size: 147,
t: new Date().getTime()
},
responseType: 'arraybuffer'
})
}Copy the codeThe appID size and t parameters can be captured. Notice that I’m using arrayBuffer for responseType, which is json by default. Buffer is mainly for us to write images locally
defaultInfo.cookies = cookieParser(result.headers['set-cookie'])
defaultInfo.cookieData = result.headers['set-cookie'];
const image_file = result.data;
await writeFile('qr.png', image_file)Copy the codeasync function writeFile(fileName, file) {
return await new Promise((resolve, reject) => {
fs.writeFile(fileName, file, 'binary', err => {
opn('qr.png')
resolve()
})
})
}Copy the codeThe first step is to write the cookieParser to parse the parameters, mainly to get the wlfSTK_SMDL, which will be used next, and then directly writeFile to write the image. Sindresorhus opN library or pretty easy to use, you can specify procedures to open pictures, files, etc
We need to listen for the scan status before we scan
async function listenScan() { let flag = true let ticket while (flag) { const callback = {} let name; Callback [name = (' jQuery + getRandomInt (100000, 999999))] = data = > {the console. The log (` ${data. The MSG | | 'sweep code is successful, '} ') if (data.code === 200) {flag = false; ticket = data.ticket } } const result = await request({ method: 'get', url: 'https://qr.m.jd.com/check', headers: Object.assign({ Host: 'qr.m.jd.com', Referer: 'https://passport.jd.com/new/login.aspx', Cookie: defaultInfo.cookieData.join('; ') }, defaultInfo.header), params: { callback: name, appid: 133, token: defaultInfo.cookies['wlfstk_smdl'], _: new Date().getTime() }, }) eval('callback.' + result.data); await sleep(1000) } return ticket }Copy the codeAt the beginning, the idea was to start a timer to poll: “Are you ok or not?” No, I will ask you again after 1 second. Here, we use the powerful function of async/await to achieve a sleep, which is more elegant than setTimeout and can be easily controlled for asynchronous processing
function sleep(ms) {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve()
}, ms)
})
}Copy the codeHere we combine the header, bring the cookie we just got, and add the host and referer to indicate where we come from and where we are going. The token in the parameter is the wLFSTK_SMDL obtained by parsing the cookie before. Callback = jSONP; callback = jSONP; callback = jSONP; At this time, MSG is not available, so under the customization, other states have MSG, directly output OK, scan the code successfully, we need to get ticket, which is understood from the literal understanding, big brother, you have got the ticket, and ticket is also needed when ordering, save it
At this time use your mobile phone to open jingdong sweep open the TWO-DIMENSIONAL code picture, confirm the scan code success, login with admission tickets
async function login(ticket) {
const result = await request({
method: 'get',
url: 'https://passport.jd.com/uc/qrCodeTicketValidation',
headers: Object.assign({
Host: 'passport.jd.com',
Referer: 'https://passport.jd.com/uc/login?ltype=logout',
Cookie: defaultInfo.cookieData.join('')
}, defaultInfo.header),
params: {
t: ticket
},
})
defaultInfo.header['p3p'] = result.headers['p3p']
return defaultInfo.cookieData = result.headers['set-cookie']
}Copy the codeThere’s nothing to be said for this step, so you have the ticket, you login successfully, you get the p3p parameter and you update the cookie and you have a legitimate identity, right
Once you have the identity, you can go to the get product page. This step requires you to put together the information of the three requests
Get the HTML for the product page
function goodInfo(goodId) {
const stockLink = `http://item.jd.com/${goodId}.html`
return request({
method: 'get',
url: stockLink,
headers: Object.assign(defaultInfo.header, {
cookie: defaultInfo.cookieData.join('')
}),
responseType: 'arraybuffer'
})
}Copy the codeGet the price of the item
async function goodPrice(stockId) {
const callback = {}
let name;
let price;
callback[name = ('jQuery' + getRandomInt(100000, 999999))] = data => {
price = data
}
const result = await request({
method: 'get',
url: 'http://p.3.cn/prices/mgets',
headers: Object.assign(defaultInfo.header, {
cookie: defaultInfo.cookieData.join('')
}),
params: {
type: 1,
pduid: new Date().getTime(),
skuIds: 'J_' + stockId,
callback: name,
},
})
eval('callback.' + result.data)
return price
}Copy the codeThe state of receiving the goods
async function goodStatus(goodId, areaId) {
const callback = {}
let name;
let status
callback[name = ('jQuery' + getRandomInt(100000, 999999))] = data => {
status = data[goodId]
}
const result = await request({
method: 'get',
url: 'http://c0.3.cn/stocks',
headers: Object.assign(defaultInfo.header, {
cookie: defaultInfo.cookieData.join('')
}),
params: {
type: 'getstocks',
area: areaId,
skuIds: goodId,
callback: name,
},
responseType: 'arraybuffer'
})
const data = iconv.decode(result.data, 'gb2312')
eval('callback.' + data)
return status
}Copy the codeFinally Promise. All wave away
async function runGoodSearch() { let flag = true while (flag) { const all = await Promise.all([goodPrice(defaultInfo.goodId), goodStatus(defaultInfo.goodId, defaultInfo.areaId), goodInfo(defaultInfo.goodId)]) const body = $.load(iconv.decode(all[2].data, 'gb2312')) outData.name = body('div.sku-name').text().trim() const cartLink = body('a#InitCartUrl').attr('href') outData.cartLink = cartLink ? 'http:' + cartLink : Price = all[0][0].p outdata. stockStatus = all[1]['StockStateName'] outdata. time = formatDate(new Date(), 'MM - dd yyyy - hh: MM: ss') console. The log () the console. The log (` goods details -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- `) console. The log (` time: ${outdata.time} ') console.log(' merchandise name: ${outdata.name} ') console.log(' price: ${outdata.price} ') console.log(' status: ${outdata.stockStatus} ') console.log(' Merchandise Connection: ${outdata.link} ') console.log(' Purchase connection: ${outdata.cartlink} ') const statusCode = all[1]['StockState'] // if (+statusCode === 33) {flag = false } else { await sleep(defaultInfo.time) } } }Copy the codeHere to parse dom, $is cheerio, known as the Node version of jQuery, but if you parse it directly, it will be messy, first transcoding, transcoding magic device appears iconV-Lite, the rest is jQuery operation, haven’t written jQuery for a long time, it is still so smooth to write
The goodId in defaultInfo is the id of the item, as we will see below
AreaId is the information corresponding to the region. After all, the inventory of each city is different
Jingdong shopping process Shopping cart first go round, and then start to order payment, we add to the cart when there are goods
Async function addCart() {console.log() console.log(' start to addCart ') const result = await request({method: 'get', url: outData.cartLink, headers: Object.assign(defaultInfo.header, { cookie: defaultInfo.cookieData.join('') }), }) const body = $.load(result.data) const addCartResult = body('h3.ftx-02') if (addCartResult) { console.log(` ${addCartresult.text ()} ')} else {console.log(' Add cart failed ')}}Copy the codeThere’s nothing to talk about. Join and start ordering
async function buy() { const orderInfo = await request({ method: 'get', url: 'http://trade.jd.com/shopping/order/getOrderInfo.action', headers: Object.assign(defaultInfo.header, { cookie: defaultInfo.cookieData.join('') }), params: { rid: new Date().getTime(), }, responseType: 'arraybuffer' }) const body = $.load(orderInfo.data) const payment = body('span#sumPayPriceId').text().trim() const sendAddr = body('span#sendAddr').text().trim() const sendMobile = body('span#sendMobile').text().trim() console.log() The console. The log (` order details -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- `) console. The log (` order total amount: ${payment} ') console.log(' ${sendAddr} ') console.log(' ${sendMobile} ') console.log() console.log(' Start order ') const result = await request({ method: 'post', url: 'http://trade.jd.com/shopping/order/submitOrder.action', headers: Object.assign(defaultInfo.header, { cookie: defaultInfo.cookieData.join('') }), params: { 'overseaPurchaseCookies': '', 'submitOrderParam.btSupport': '1', 'submitOrderParam.ignorePriceChange': '0', 'submitOrderParam.sopNotPutInvoice': 'false', 'submitOrderParam.trackID': defaultInfo.ticket, 'submitOrderParam.eid': defaultInfo.eid, 'submitOrderParam.fp': defaultInfo.fp, }, }) if (result.data.success) {console.log(' order succeeded, order no. ${result.data.orderID} ') console.log(' Please go to the store to pay in time, ${result.data.message} ')} else {console.log(' order failed,${result.data.message} ')}}Copy the codeActually here post trade.jd.com/shopping/or… This is ok, the previous request is the order page take the order information display, there will be two points of attention
-
Jingdong order is to place an order for all the goods in the shopping cart, regardless of the quantity. For example, you already have one piece of this product in your shopping cart, so the shopping cart now has two pieces of this product after the previous process is completed. After placing an order, you placed two pieces of this product
-
Submitorderparam.trackid submitOrderParam.eid submitOrderParam.fp submitOrderParam.fp So where do Eids and FP come from? The answer is the login page, but there is a pit where the dom element returned from the request page does not work. It can only be accessed through the browser, which is also easy to do. Node has Phantomjs, but here I use puppeteer from Chrome
Puppeteer is also simple to use and is a Node-based Headless Chrome tool
Puppeteer.launch ().then(async browser => {console.log(' initialization complete, start to fetch the page ') const page = await browser.newpage (); await page.goto('https://passport.jd.com/new/login.aspx'); Await sleep(1000) console.log(' page fetched complete, Start page ') const inputs = await page. The evaluate (res = > {const result = document. QuerySelectorAll (' input 'const data = {} for (let v of result) { switch (v.getAttribute('id')) { case 'token': data.token = v.value break case 'uuid': data.uuid = v.value break case 'eid': data.eid = v.value break case 'sessionId': data.fp = v.value break } } return data }) Object.assign(defaultInfo, inputs) await browser.close(); Console. log(' Page parameters in hand, Close the browser ') to the console. The log () to the console. The log (' -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ') to the console. The log (' request and yards) console. The log (' ------------------------------------- ') console.log() })Copy the codePuppeteer first launches and then generates a browser instance. We use Browser to create a new page to run our url, and we can manipulate the DOM in the Evaluate method. The code above is also very simple and easy to read
Now basically an automatic order function is complete, and then extend the command line parameters
const args = require('yargs').alias('h', 'help') .option('a', { alias: 'area', demand: true, describe: Option ('g', {alias: 'good', demand: true, describe: describe). Option ('t', {alias: 'time', describe: describe 'query interval ms', default: '10000'}). Option ('b', {alias: 'buy', describe:' whether to place an order ', default: true}). Example ('node index.js -a 2_2830_51810_0 -g 5008395'). Argv;Copy the codeHere I gave two required parameters and two optional parameters, -a required, region number, -g required, item number, -t interval between item query, default is 10s, -b whether to automatically purchase, default is purchased, Boolean, yargs is pretty good, You can also use TJ commander, which is the same
The complete code can be viewed in the project address below
Project address to find a star