This paper mainly focuses on the thinking of using canvas card in a large number of projects, and then implementing a server-side painting card function to reduce business development costs. This article first introduces the server-side picture card logic, and another article introduces the public picture card management background (update later…). .
The process is introduced
- The client or server requests the drawData interface (interface parameters: {drawData: drawData, weight: weight, cbApi: callback interface, userId, taskId}).
- Parsing parameters are stored in Redis.
- Determine whether any task is currently in execution. If not, create a Page instance and get the data in Redis at the same time. If so, add the corresponding task to draw the card waiting.
- After drawing card successfully, call callback interface, return drawing card data.
The above is the general process, and the following is the flow chart:
The Phantom. The js is introduced
Phantomjs implements a webKit browser with no interfaces. Although there is no interface, dom rendering, JS running, network access, canvas/ SVG drawing and other functions are very complete, which are widely used in page grabbing, page output, automatic testing and other aspects.
A simple example
// test.js
var page = require('webpage').create(),
system = require('system'),
address;
if (system.args.length === 1) {
phantom.exit(1);
} else {
address = system.args[1];
page.open(address, function (status) {
console.log(page.content);
phantom.exit();
});
}
Copy the coderun
phantomjs ./test.js baidu.com
Phantomjs provides many apis, so I won’t go into the details of the PhantomJS documentation here
Project structures,
The project structure uses NestJS + typescript + Redis + mongodb as the server and React + typescript + mobx + ANTD as the management background.
1, Nestjs
Nest is a framework for building efficient, scalable Node.js server-side applications. It uses progressive JavaScript, has built-in and full TypeScript support (but still allows developers to write code in pure JavaScript), and combines elements of OOP (object-oriented programming), FP (functional programming), and FRP (functional response programming). Reference documentation for installing and running nestjs
2. Redis builds
At the beginning of the project, cache-manager-Redis-store was used as data cache and redis operation method was provided. Drawing cards now uses redis’s List API directly.
CacheConfigService requires the CacheOptionsFactory interface to provide configuration options
3. Business code
3.1 Service Directory Structure
- Task.config.service. ts provides initialization of PhantomJS
- The task.controller.ts controller is responsible for processing incoming requests and returning responses to the client.
- Task.service. ts processes the card service logic
- Task. The module. Ts modules
3.2 Initializing PhantomJS
First, create a PhantomTask class that hosts all configuration listening information for Phantomjs initialization.
* @export * @class PhantomTask */ export class PhantomTask {// page object public WebPage // phantom object public phantomInstance: PhantomJS private num: number = 0 private event: WebPage EventEmitter constructor() { if (! this.event) { this.event = new EventEmitter() } } ... }Copy the codeProvides an external initialization API; Initialize Phantom configuration and listen Phantom related methods; After the successful drawing of the card, the data is obtained by listening to the onAlert method to obtain the alert popover data. This.event. emit triggers the custom event to transmit the card data. Note that Phantomjs does not support ES6 syntax
@memberof PhantomTask */ public async initTask() {if (! This.clientpage) {try {this.phantomInstance = await create(['--ignore-ssl-errors=yes']) // create page this.clientPage = Await this. PhantomInstance. CreatePage () / / start time let _serverRequestStartTime: [number, number] = process.hrtime() // Whether to execute javascript this.clientPage. Setting ('javascriptEnabled', This.clientpage. setting('loadImages', true) this.clientPage.on('onLoadStarted', () => { console.log('[phantomjs task log]: ') _serverRequestStartTime = process.hrTime ()}) this.clientPage. On ('onLoadFinished', (status) => { if (Object.is(status, 'success')) { const _serverRequestEndTime: [number, number] = process.hrtime() const ms = (_serverRequestEndTime[0] - _serverRequestStartTime[0]) * 1e3 + (_serverRequestEndTime[1] - _serverRequestStartTime[1]) * 1e-6 console.log('[phantomjs task log]: Else {console.log('[phantomjs task log]: Alert this.clientPage. On ('onAlert', (data) => {this.event. Emit ('imageData', Data) console.log('onAlert:========', data)}) this.clientPage. On ('onError', (MSG, Trace) => {console.error('onError ------ ', MSG, trace)}) (msg, lineNum, sourceId) => { console.log('onConsoleMessage ------ ', msg, lineNum, SourceId)}) // Trigger this.clientPage. On ('onInitialized', () => {console.log('[phantomjs task log]page created after ')}) // Listen for URL changes this.clientPage. (targetUrl) => {console.log('[Phantomjs task log] listener url', targetUrl)}) Es6 code does not support const result = await this.clientPage.open(' enter the page address ') const Content = await This.clientpage.property ('content') // console.log(content) console.log('[phantomjs task log] completes page loading ', result) } catch (error) { console.log('page error:', Error) enclosing phantomInstance && enclosing phantomInstance. The exit () enclosing phantomInstance = null enclosing clientPage = null number / / limit the if (this.num < 10) {this.num++ setTimeout(() => {this.initTask}, 2000)} else {console.error(' this. Num < 10) {this.num++ setTimeout(() => {this.inittask}, 2000)} else {console.error(' This. ')}}}}Copy the codeThis method is called when the drawing card data is obtained, triggering the page drawing card function and monitoring the data acquisition after the completion of the drawing card. Evaluate function (which operates on the window object). The EVALUATE function can also be triggered directly using the ES5 wrapper function. Considering the general maintainability of the code (maintaining a set of code), here we call the get draw card management background page and draw the card by triggering the window.initdraw () function.
/** * triggers the page draw card function and listens to the data after the draw card is completed. *@param {*} cardData
* @returns
* @memberof PhantomTask* /
public async createShareCard(cardData): Promise<string> {
if (!this.clientPage) {
console.error('[Phantomjs task Error]: The page was not initialized successfully! ')
return null
}
return new Promise((resolve, reject) = > {
// Get the parameters through the Alert popup, and get the data through the imageData event
this.event.once('imageData'.(data) = > {
console.log('[Phantomjs task log]: Image production completed ')
resolve(data)
})
// page displays the page context to perform function
this.clientPage.evaluate(function(data) {
// Place the page drawing card function directly under the window object.
// window.initDraw(data)
console.log('Provides window operations, such as triggering card drawing')
// Get text can be returned directly, external can get text directly
// return document.querySelector('.title').innerText;
// Es6 syntax is not supported
// const test = () => {
// console.log(1)
// }
// test()
}, [cardData])
})
}
Copy the code3.3 Processing Request Services
TaskService contains high and low tasks. Add the task to the List of high or low tasks according to the weight.
/** * Picture card, screenshot service *@export
* @class TaskService
* @extends {ServiceExt}* /
@Injectable()
export class TaskService extends ServiceExt {
private phantomTask: PhantomTask
// High task key
private HEIGH_CARD_KEY: string = 'HEIGH_CARD_KEY'
// Low task key
private SHARE_CARD_DATA: string = 'SHARE_CARD_DATA'
/ / the current key
private currentRedisKey: string = this.SHARE_CARD_DATA
// Used to store the current card
private currentTaskData: any = null
// Determine if any new tasks are added
private isJoin: boolean = false
// Used to indicate whether the page instance is cleared
private isClear: boolean = false
/ / the countdown
private timer: any = null
constructor(
private readonly httpService: HttpService, //
private readonly cacheService: CacheService,
) {
super(a)this.phantomTask = new PhantomTask()
}
...
}
Copy the codeProcess the interface request data and generate a unique identifier according to the current time for judging the subsequent card drawing task. The data is stored in Redis by level. Create if no task is currently in progress.
/** * draw card *@param {*} body
* @memberof TaskService* /
public async cardTask(body: TaskDrawModel) {
this.isJoin = true
this.isClear && (this.isClear = false)
// Generate a unique identifier for subsequent logical judgment
body.taskKey = Date.now() + ' ' + Math.round(Math.random() * 1000)
const data = JSON.stringify(body).replace(/\u2028/g.' ')
// Add weights to the corresponding List data
const key = Object.is(body.weight, 'max')?this.HEIGH_CARD_KEY : this.SHARE_CARD_DATA
try {
// Add to redis
const status = await this.cacheService.lpush(key, data)
setTimeout(async() = > {// If no task exists, create phantomTask
if (!this.phantomTask.phantomInstance) {
await this.phantomTask.initTask()
// Update task key
this.currentRedisKey = key
this.taskRun(true)}this.isJoin = false
}, 10)
return { code: 200.msg: 'Add task to success'}}catch (error) {
return { msg: 'Failed to add task to queue'}}}Copy the codePromise.race([]) accepts two promises, the first for the current task to run and the second to prevent the task from getting stuck affecting all card tasks.
/** * Get the Redis data length to determine whether the task exists. *@private
* @param {boolean} [flag]
* @memberof TaskService* /
private async taskRun(flag? : boolean) {
try {
// Check whether the current high task exists
if (this.currentRedisKey ! = =this.HEIGH_CARD_KEY && ! flag) {await this.heightTask()
}
// Get the task length
const len = await this.cacheService.llen(this.currentRedisKey)
if (len > 0) {
// tslint:disable-next-line:prefer-const
console.log('[task log (share card)]: key='.this.currentRedisKey , 'Current task allowance', len)
Promise.race([
new Promise(async (resolve, reject) => {
try {
// Get the card data to trigger the card
await this.createTask()
clearTimeout(this.timer)
resolve(' ')}catch (error) {
reject(error)
}
}),
new Promise(async (resolve, reject) => {
try {
// Prevent one task from getting stuck and affecting all tasks
await this.taskOvertime()
} catch (error) {
reject(error)
}
}),
]).then(() = > {
this.currentTaskData = null
if (this.isClear && !this.isJoin) {
console.log('[task log (share card)], all tasks completed, no recursion ')}else {
this.taskRun()
}
}).catch((error) = > {
const backData = JSON.stringify(this.currentTaskData) + '\n'
console.error('Task failed, data=', backData, ' error info=', error)
this.currentTaskData = null
this.taskRun()
})
} else {
this.clearTask()
console.log('[Task log (Share card)]: There is no data now!! ')}}catch (error) {
console.error('[task log (Share card)]: Failed to query number of remaining tasks', error)
console.log('[task log (share card)]: try to query again... ')
setTimeout(() = > {
this.taskRun()
}, 1000)}}Copy the codeCheck whether a high task exists, and run the high task first
/** * Check whether the high task exists, and run the high task first@private
* @memberof TaskService* /
private async heightTask() {
const len = await this.cacheService.llen(this.HEIGH_CARD_KEY)
this.currentRedisKey = len > 0 ? this.HEIGH_CARD_KEY : this.SHARE_CARD_DATA
}
/** * Read data to start task *@private
* @memberof TaskService* /
private async createTask() {
try {
const _serverRequestStartTime: [number, number] = process.hrtime()
const result = await this.cacheService.rpop(this.currentRedisKey)
const _serverRequestEndTime: [number, number] = process.hrtime()
const ms = (_serverRequestEndTime[0] - _serverRequestStartTime[0]) * 1e3 + (_serverRequestEndTime[1] - _serverRequestStartTime[1]) * 1e-6
console.log('[task log (share card)]:, ms, 'ms')
if (result) {
console.log('[task log (Share card)]: Start task execution --', result)
const shareData = JSON.parse(result)
// The current picture of the card is being drawn
this.currentTaskData = shareData
const taskKey = this.currentTaskData.taskKey
// Trigger the front drawing card
const imageUrl = await this.phantomTask.createShareCard(shareData.drawData)
console.log(`taskKey=${taskKey} --- shareData.taskKey=${shareData.taskKey}`)
// taskKey Checks whether tasks are consistent to prevent task timeout
if (taskKey === shareData.taskKey) {
console.log('Draw card successfully started request callback interface ---------')
this.requestCallback(imageUrl, shareData)
} else {
console.log('Task timed out, no push card processing')}}else { // If no task exists in the queue
console.log('[Task log (share card)] no tasks in queue, data = null')
this.clearTask()
}
} catch (error) {
console.log('[task log (share card)] ', error)
// Throw an exception
throw new Error(error)
}
}
/** * is used to handle the task timeout problem and block the card progress *@private
* @memberof TaskService* /
private taskOvertime() {
const _serverRequestStartTime: [number, number] = process.hrtime()
return new Promise((resolve, reject) = > {
this.timer = setTimeout(async() = > {const _serverRequestEndTime = process.hrtime()
const ms = (_serverRequestEndTime[0] - _serverRequestStartTime[0]) * 1e3 + (_serverRequestEndTime[1] - _serverRequestStartTime[1]) * 1e-6
console.error('[task log (Share card)]:, ms, 'ms')
if (!this.currentTaskData.hadCached) {
console.log('Timeout task, push to Redis and retry, data='.JSON.stringify(this.currentTaskData))
this.currentTaskData.hadCached = true
await this.cacheService.lpush(this.HEIGH_CARD_KEY, JSON.stringify(this.currentTaskData))
} else {
console.log('Task has already been retried, task will not be retried')}clearTimeout(this.timer)
reject('Create card timeout. ')},5000)})}/** * Clear task *@private
* @memberof TaskService* /
private async clearTask() {
const len: number = await this.cacheService.llen(this.SHARE_CARD_DATA)
const len1: number = await this.cacheService.llen(this.HEIGH_CARD_KEY)
console.log('[task log] retrieves all tasks: SHARE_CARD_DATA===${len}, HEIGH_CARD_KEY=== ${len1}, whether new tasks are currently added: isJoin=The ${this.isJoin }`)
// If all tasks are missing and no new tasks are added, clear
if (len === 0 && len1 === 0&&!this.isJoin) {
this.phantomTask.phantomInstance.exit()
this.phantomTask.phantomInstance = null
this.phantomTask.clientPage = null
this.isClear = true
} else {
// If no new task is added, query the cache, otherwise get the latest task
if (len > 0) {
this.currentRedisKey = this.SHARE_CARD_DATA
}
if (len1 > 0) {
this.currentRedisKey = this.HEIGH_CARD_KEY
}
}
}
Copy the codeThe httpService callback interface returns the result of a successful draw.
/** * Callback after drawing card success *@private
* @param {string} imgUrl
* @param {TaskDrawModel} data
* @memberof TaskService* /
private requestCallback(imgUrl: string, data: TaskDrawModel) {
const parmas = {
userId: data.userId,
taskId: data.taskId,
imgUrl
}
this.httpService.post(data.cbApi, {... parmas}).pipe( retry(3), // Retry three times
catchError(val= > of('fail'))
).subscribe((val) = > {
if(val ! = ='fail') {
console.log('Callback successful')}else {
console.error('Callback failed')}}}Copy the codeThe last
That’s the end of this article. I hope it helps. This article mainly explains how to realize the node layer service drawing card function, the next chapter introduces the drawing card management background business.
Xiaobian for the first time to write an article writing style is limited, untalented and shallow, the article if there is wrong, hope to inform.