preface
Is there a lot of people like me have such a worry, every day there are endless requirements, endless bugs, every day repetitive, tedious business code, worried about their own technology growth.
In fact, on the other hand, all the front-end technology we learn is for the business, so why don’t we find a way to use the front-end technology to do something for the business? This will not only solve the business problem, but also get you out of the trouble of having to write repetitive and tedious code every day.
This article is mainly for the author in view of some business problems in the current team, the realization of an automated deployment platform technical scheme.
The original address
background
At the beginning of last year, as there was no front-end in the team, I happened to be the first and only FE recruited, so I took over a JSSDK project that had been maintained by the back end. In fact, I couldn’t call it a project. When I took over, it was just a fat script with more than 2000 lines of code, without any traces of engineering.
The business requirements
The JSSDK is mainly used to write the appKey in the JSSDK after assigning appKey to the service side at the back end. After uploading the appKey to the CDN, the script of data collection service is provided for the service side.
Some students may wonder why appKey is not passed into the JSSDK as a parameter like some normal SDKS, so that all business parties can use the same JSSDK without providing a JSSDK for each business party. In fact, I had the same idea at the beginning, so I proposed my idea to my leader, which was rejected. The reasons are as follows:
- If the appKey is imported as a parameter, the access cost for the service increases, and the appKey may be incorrectly filled.
- After accessing the JSSDK, the client expects that each ITERATION of the JSSDK version will be insensitive to the client (that is, the version iteration is an overlay release). If all clients use the same JSSDK, each iteration of the JSSDK version will affect all clients at one time, increasing risks.
Since my leader is now mainly responsible for product promotion and often deals with the business side, he may be more able to consider problems from the perspective of the business side. Therefore, it is understandable that my leader chose to sacrifice the maintenance cost of the project to reduce the access cost of SDK and avoid risks.
Well, if we can’t change the status quo, we have to adapt to it.
Project (
Then for the fat script without any engineering, I need to do the following every time I add a new business side:
- Open a fat script and JSSDK access document and copy a new one.
- Find the appKey to be allocated to the back end, find the corresponding appKey line of code to manually modify.
- Manually mix up the modified script and upload it to the CDN.
- Modify the CDN address in the JSSDK access document, save it, and send it to the service party.
The whole process is manual, relatively tedious and prone to errors, with scripts and access documentation checked every time.
In view of the above situation, we get the problems we need to solve:
- How to quickly output a new JSSDK and access document for a new business?
- How to quickly obfuscate the new JSSDK and upload it to the CDN.
Automation scheme
Before introducing the scheme, first take a screenshot of the platform to have an intuitive understanding:
SDK automatic deployment platform mainly realizes JSSDK compilation, release test (online preview), upload CDN functions.
The server stack includes:
- Framework Express
- Hot update nodemon
- Dependency injection Awilix
- Data persistence sequelize
- The deployment of pm2
Decorator + Vue -property-decorator + vuex-class
Project construction reference: Vue+Express+Mysql full stack initial experience
The automatic deployment platform mainly relies on GIT + local environment + private NPM source + MYSQL to communicate and interact with each other to complete automatic deployment.
Main effects achieved: After the local environment pulls the git repository code, the requirements are developed, and a Rollup SDK compiler package is released to the private NPM repository after completion. The automated deployment platform installs the SDK of the specified version in the project directory and backs it up locally. When the SDK is compiled, select the SDK compiler of the specific version Rollup. At the same time, JSSDK access documents are automatically generated and packaged into a Release package with a description file. When uploading to CDN, the corresponding information of the description file is written into MYSQL for saving.
Version management
Since the JSSDK was originally just a script, we had to engineer the project so that version management could be completed so that it could be released quickly, rolled back, and quickly cut losses.
First, we need to engineer the project, use Rollup for module management, and when sending NPM package, input various parameters (such as appKey) and output a Rollup Complier function. Then use rollup-plugin-replace to replace specific parameters in the code at compile time.
Lib /build.js, an entry file for JSSDK packages, which is used during SDK compilation
import * as rollup from 'rollup';
const replace = require('rollup-plugin-replace');
const path = require('path');
const pkgPath = path.join(__dirname, '.. '.'package.json');
const pkg = require(pkgPath);
const proConfig = require('./proConfig');
function getRollupConfig(replaceParams) {
const config = proConfig;
// Inject system variables
const replacePlugin = replace({
'__JS_SDK_VERSION__': JSON.stringify(pkg.version),
'__SUPPLY_ID__': JSON.stringify(replaceParams.supplyId || '7102'),
'__APP_KEY__': JSON.stringify(replaceParams.appKey)
});
return {
input: config.input,
output: config.output,
plugins: [
...config.plugins,
replacePlugin
]
};
};
module.exports = async function (params) {
const config = getRollupConfig({
supplyId: params.supplyId || '7102'.appKey: params.appKey
});
const {
input,
plugins
} = config;
const bundle = await rollup.rollup({
input,
plugins
});
const compiler = {
async write(file) {
await bundle.write({
file,
format: 'iife'.sourcemap: false.strict: false}); }};return compiler;
};
Copy the codeIn an automated deployment platform, install the JSSDK package using ShellJS:
import {route, POST} from 'awilix-express';
import {Api} from '.. /framework/Api';
import * as shell from 'shell';
import * as path from 'path';
@route('/supply')
export default class SupplyAPI extends Api {
// some code
@route('/installSdkVersion')
@POST()
async installSdkVersion(req, res) {
const {version} = req.body;
const pkg = `@baidu/xxx-js-sdk@${version}`;
const registry = 'http://registry.npm.baidu-int.com';
shell.exec(`npm i ${pkg} --registry=${registry}`, (code, stdout, stderr) => {
if(code ! = =0) {
console.error(stderr);
res.failPrint('npm install fail');
return;
}
// SDK package backup directory
const sdkBackupPath = this.sdkBackupPath;
const sdkPath = path.resolve(sdkBackupPath, version);
shell.mkdir('-p', sdkPath).then((code, stdout, stderr) = > {
if(code ! = =0) {
console.error(stderr);
res.failPrint(`mkdir \`${sdkPath}\` error.`);
return;
}
const modulePath = path.resolve(process.cwd(), 'node_modules'.'@baidu'.'xxx-js-sdk');
// Copy the installed files for subsequent use
shell.cp('-rf', modulePath + '/., sdkPath).then((code, stdout, stderr) = > {
if(code ! = =0) {
console.error(stderr);
res.failPrint(`backup sdk error.`);
return;
}
res.successPrint(`${pkg} install success.`); }); })}); }}Copy the codeRelease the package
Release package is the compressed package we need to prepare before uploading to CDN. Therefore, after packaging the JSSDK, we need to generate files such as access documentation, JSSDK DEMO preview page, JSSDK compilation results, and description files.
First, the packing function is as follows:
import {Service} from '.. /framework';
import * as fs from 'fs';
import path from 'path';
import _ from 'lodash';
export default class SupplyService extends Service {
async generateFile(supplyId, sdkVersion) {
// The database queries the CDN name of the corresponding service party
const [sdkInfoErr, sdkInfo] = await this.supplyDao.getSupplyInfo(supplyId);
if (sdkInfoErr) {
return this.fail('Server error'.null, sdkInfoErr);
}
const {appKey, cdnFilename, name} = sdkInfo;
// The data to be replaced
const data = {
name,
supplyId,
appKey,
'sdk_url': `https://***.com/sdk/${cdnFilename}`
};
try {
/ / compile JSSDK
const sdkResult = await this.buildSdk(supplyId, appKey, sdkVersion);
// Generate access files
const docResult = await this.generateDocs(data);
// Generate a preview DEMO HTML file
const demoHtmlResult = await this.generateDemoHtml(data, 'sdk-demo.html'.'JSSDK- Access page -${data.name}.html`);
// Generate the release package description file
const sdkInfoFileResult = await this.writeSdkVersionFile(supplyId, appKey, sdkVersion);
const success = docResult && demoHtmlResult && sdkInfoFileResult && sdkResult;
if (success) {
// release target directory
const dir = path.join(this.releasePath, supplyId + ' ');
const fileName = `${supplyId}-${sdkVersion}.zip`;
const zipFileName = path.join(dir, fileName);
// Compress all result files
const zipResult = await this.zipDirFile(dir, zipFileName);
if(! zipResult) {return this.fail('Package failed');
}
// Return the zip package for download
return this.success('Packed successfully', {
url: ` /${supplyId}/${fileName}`
});
} else {
return this.fail('Package failed'); }}catch (e) {
return this.fail('Package failed'.null, e); }}}Copy the codeCompile JSSDK
JSSDK compilation is very simple, just need to load the corresponding version of JSSDK Compiler function, and then pass the corresponding parameters into the Compiler function to get a Rollup Compiler, and then write the Compiler result into the Release path.
export default class SupplyService extends Service {
async buildSdk(supplyId, appKey, sdkVersion) {
try {
const sdkBackupPath = this.sdkBackupPath;
// Load the Rollup compiler for the corresponding version of the backup JSSDK package
const compileSdk = require(path.resolve(sdkBackupPath, sdkVersion, 'lib'.'build.js'));
const bundle = await compileSdk({
supplyId,
appKey: Number(sdkInfo.appKey)
});
const releasePath = path.resolve(this.releasePath, supplyId, `${supplyId}-sdk.js`);
// Rollup Compiler compiles the result to the release directory
await bundle.write(releasePath);
return true;
} catch (e) {
console.error(e);
return false; }}}Copy the codeGenerating access Documents
The principle is simple: use JSZip, open the access document template, replace the special characters in the template with Docxtemplater, and then regenerate the DOC file:
import Docxtemplater from 'docxtemplater';
import JSZip from 'JSZip';
export default class SupplyService extends Service {
async generateDocs(data) {
return new Promise(async (resolve, reject) => {
if (data) {
// Read access file, replace appKey, CDN path
const supplyId = data.supplyId;
const docsFileName = 'sdk-doc.docx';
const supplyFilesPath = path.resolve(process.cwd(), 'src/server/files');
const content = fs.readFileSync(path.resolve(supplyFilesPath, docsFileName), 'binary');
const zip = new JSZip(content);
const doc = new Docxtemplater();
// Replace '[[' prefix and']] 'suffixes
doc.loadZip(zip).setOptions({delimiters: {start: '[['.end: ']] '}});
doc.setData(data);
try {
doc.render();
} catch (error) {
console.error(error);
reject(error);
}
// Generate DOC's buffer
const buf = doc.getZip().generate({type: 'nodebuffer'});
const releasePath = path.resolve(this.releasePath, supplyId);
// Create the target directory
shell.mkdir(releasePath).then((code, stdout, stderr) = > {
if(code ! = =0 ) {
resolve(false);
return;
}
// Write the replacement to the release path
fs.writeFileSync(path.resolve(releasePath, ` JSSDK - documentation -${data.name}.docx`), buf);
resolve(true);
}).catch(e= > {
console.error(e);
resolve(false); }); }}); }}Copy the codeGenerate a preview DEMO page
Similar to the principle of access document generation, open a DEMO template HTML file, replace the internal characters, and regenerate the file:
export default class SupplyService extends Service {
generateDemoHtml(data, file, toFile) {
return new Promise((resolve, reject) = > {
const supplyId = data.supplyId;
// The data to be replaced
const replaceData = data;
// Open the file
const content = fs.readFileSync(path.resolve(supplyFilesPath, file), 'utf-8');
// The string replaces the contents of the '{{' prefix and'}} 'suffix
const replaceContent = content.replace(/{{(.*)}}/g, (match, key) => {
return replaceData[key] || match;
});
const releasePath = path.resolve(this.releasePath, supplyId);
// Write to the file
fs.writeFile(path.resolve(releasePath, toFile), replaceContent, err => {
if (err) {
console.error(err);
resolve(false);
} else {
resolve(true); }}); }); }}Copy the codeGenerate the Release package description file
Some parameters currently packaged are stored in a file and packaged together into the Release package. The function is very simple, which is used to describe some parameters currently packaged, so as to facilitate the recording of which SDK version is currently online when CDN is launched
export default class SupplyService extends Service {
async writeSdkVersionFile(supplyId, appKey, sdkVersion) {
return new Promise(resolve= > {
const writePath = path.resolve(this.releasePath, supplyId, 'version.json');
// Release describes data
const data = {version: sdkVersion, appKey, supplyId};
try {
// Write to the release directory
fs.writeFileSync(writePath, JSON.stringify(data));
resolve(true);
} catch (e) {
console.error(e);
resolve(false); }}); }}Copy the codePackage all file results
Package the previously generated JSSDK compilation results, access documents, preview DEMO page files, and description files using archive:
export default class SupplyService extends Service {
zipDirFile(dir, to) {
return new Promise(async (resolve, reject) => {
const output = fs.createWriteStream(to);
const archive = archiver('zip');
archive.on('error', err => reject(err));
archive.pipe(output);
const files = fs.readdirSync(dir);
files.forEach(file= > {
const filePath = path.resolve(dir, file);
const info = fs.statSync(filePath);
if(! info.isDirectory()) { archive.append(fs.createReadStream(filePath), {'name': file }); }}); archive.finalize(); resolve(true); }); }}Copy the codeCDN deployment
Most of the files uploaded to CDN are like CDN source station push files, and we have mounted NFS on the machine of my automatic deployment platform, that is, I only need to locally copy the JSSDK files to the shared directory to realize THE CDN file upload.
export default class SupplyService extends Service {
async cp2CDN(supplyId, fileName) {
// Read the description file
const sdkInfoPath = path.resolve(this.releasePath, ' ' + supplyId, 'version.json');
if(! fs.existsSync(sdkInfoPath)) {return this.fail('Release description file is missing, please repackage ');
}
const sdkInfo = JSON.parse(fs.readFileSync(sdkInfoPath, 'utf-8'));
sdkInfo.cdnFilename = fileName;
// Copy the file to the shared directory
const result = await this.cpFile(supplyId, fileName, false);
// Upload succeeded
if (result) {
// synchronize data from the Release package description file to MYSQL
const [sdkInfoErr] = await this.supplyDao.update(sdkInfo, {where: {supplyId}});
if (sdkInfoErr) {
return this.fail('JSSDK info recording failed, please try again '.null, jssdkInfoResult);
}
return this.success('Upload successful', {url})
}
return this.fail('Upload failed'); }}Copy the codeProject results
The benefits of the project are still obvious, which essentially solves the problem we need to solve:
- Completed the project engineering, automatic generation of JSSDK and access documents.
- Obfuscation is automated during compilation and one-click upload to CDN is implemented.
Save manual upload and paste code time, greatly improve the work efficiency.
This project was a tool project I completed in my spare time in the first half of 2019. Later, the Leader paid attention to it and upgraded the tool to a platform, integrating many business-related configurations on the platform. This is how I got my KPI in the first half of 2019, ha ~~~
conclusion
Or maybe it works for every business
- Understand the background of the business
- Identify business pain points
- Look for solutions and take initiative to implement them
- To solve the problem
In fact, the pain points in each project are generally low performance and low efficiency of XX, which are relatively easy to find. At this time, it is only necessary to take the initiative to find solutions and promote the implementation.
Front-end technology is inseparable from the business, technology always serves the business, leaving the business of the technology, it is completely no foothold technology, completely meaningless technology. Therefore, in addition to writing the page, the use of front-end page tool, automation, so as to promote the platform is also a good foothold.