This site is licensed under the “CC BY 4.0 international” license, welcome to reprint, or modify the use, but need to indicate the source. Signature 4.0 International (CC BY 4.0)
Author: Su Yang
Creation time: August 31, 2018 statistical word count: 4273 words reading time: 9 minutes to read this article links: soulteary.com/2018/08/31/…
If node.js is not used to write services
It’s been a long time since I’ve covered some language related uses in my blog, so I took a break to take a look at the code repository and discuss some interesting things node.js can do besides writing business scripts and doing build runs.
The glue in continuous integration
In continuous integration, occasionally there will be a need to parse API results, or read a file, and get the specified data from the file. When it is difficult to accomplish the requirements with a regular shell, using Node as a binder to get data is a good solution compared to Python. Because the written code will be much simpler.
The sample
Let’s say you want to get a particular field in some API, and then assemble the parameters again, requesting other apis, and achieving some degree of complete automation.
Node.js fs.readFileSync method to read /dev/stdin, for example:
curl SOME_API_WITH_ARGS | node -e 'console.log(JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).data); '
Copy the codeOf course, you could save the script as a file and call it, and the result would be the same.
const fs = require("fs"); console.log(JSON.parse(fs.readFileSync("/dev/stdin").toString()).iid);
Copy the codeIf you save the above code as pipe.js, then execute the following code and get the same result.
echo data | node pipe.js
Copy the codeAssist in data clearing
When it comes to data cleaning, we will automatically think of SQL, Python and R, but sometimes, there is no need to do a lot of coding, using the built-in Shell system with Node.js will be faster in implementation and efficiency.
The sample
Once the program is built and you want to see how much of it is redundant, you need to de-redo the file. Usually we can’t simply rely on the name of the file for deduplication, so we use the file content signature to do this.
Using a statically compiled shell tool, you can quickly calculate the signature of a file:
find . | xargs -I {} shasum -a 256 {}
shasum: ./dist:
shasum: ./dist/some:
b0a635b4d6be1a6f14f25c64e3ff6fa2ccacc4e55be93d6f542184a386f748c8 ./dist/some/aea0d6c054cf3fadf5bb.js
9498cb1451fc6d02612610f109ebb29dc94957e2a2741b701443182573837bfc ./dist/some/
Copy the codeThe output will have a lot of seemingly meaningless lines, so it only takes a few lines of code to parse the file when passed to the node.js program.
const fs = require('fs');
const shasumList = fs.readFileSync('/dev/stdin').toString().split('\n').filter(l => l).filter(l => l.match(/^\w{64}\s+\S+$/));
let reduceResult = {};
shasumList.map(item => {
const [shasum, name] = item.match(/(\w+)\s+(\S+)/);
return {shasum, name};
}).forEach(item => {
reduceResult[item.shasum] = reduceResult[item.shasum] || [];
reduceResult[item.shasum].push(item.name);
});
console.log(reduceResult);
Copy the codeOf course, if you’d like, it’s also possible to write a simple directory scan and use Node.js’s crypto module to calculate the signature directly.
const fs = require('fs');
const crypto = require('crypto');
const secret = ' ';
const hash = crypto.createHmac('sha256', secret).update(fs.readFileSync(`${item.name}-${item.id}.html`).toString()).digest('hex');
Copy the codeCurrize external operations
If the title sounds familiar, when we think of Corrification, we think of encapsulating higher-order functions with fewer arguments. Similar functionality can be provided using Node.js, except that curry is not happening at the code level.
The sample
For example, I found some code snippet that was zero years old and didn’t exist in CVS version control. If I added it directly, the commit time would be the present commit time. But the files themselves provide the creation date and can be stored as raw information.
Git commit –date can be used to change the commit timestamp, but it is very difficult to use because it is in a specific format: DDD, DD MMM YYYY HH:MM:SS +8000.
It would be much easier to work with these files if there was a little script here that would automatically convert the entered time into this format.
#! /usr/bin/env node
var moment = require('moment');
var argvs = process.argv.slice(2);
console.log('input:', argvs[0]);
if(! argvs.length) { console.log('useage: tool timestamp');
process.exit(1);
}
console.log('output:\n');
if (argvs.length === 2) {
console.log('git add .');
console.log('gc -m "' + argvs[0] + '" --date="' + moment(new Date(argvs[1])).format('ddd, DD MMM YYYY HH:MM:SS +8000') + '"');
console.log('git push');
} else {
console.log('gc --amend -m "' + moment(new Date(argvs[0])).format('ddd, DD MMM YYYY HH:MM:SS +8000') + '" --date="' + moment(new Date(argvs[0])).format('ddd, DD MMM YYYY HH:MM:SS +8000') + '"');
console.log('git push -f');
}
Copy the codeOf course, if you save the script to a globally accessible path and then grant execution permissions, you can also add exec operations to the script to make the code commit automatically.
Auxiliary file management
In addition to being a tool for data management, Node.js provides a file manipulation module that can also assist us in file sorting.
The sample
After buying XX music package and XX membership, I found that the sound quality of songs is indeed much better, but the cost of sound quality is that I have to cache a large number of files in mobile devices, even using 256G mobile devices, but also can not bear the weight of years of caching. So I put the music in the equipment export, download some music from other places, also plans to put them in the same return to manage a dedicated equipment in the store, but thousands of randomly generated file name is very unfavorable to archive, fortunately, the majority of the information contained in the music files ID3, cooperate with the analytic software packages, let the program automatically they renamed and processes.
const {readdirSync, renameSync, statSync, unlinkSync, existsSync, mkdirSync} = require('fs');
const {join} = require('path');
const {inspect} = require('util');
const {parseFile} = require('music-metadata');
const {rootDir} = require('./config');
const files = readdirSync(rootDir).
filter((file) => file.endsWith('.m4a') || file.endsWith('.flac') || file.endsWith('.mp3'));
files.forEach((file) => {
const fullPath = join(rootDir, file);
parseFile(fullPath, {native: true}).then((metadata) => {
const name = `${metadata.common.title.trim()} - ${metadata.common.artist.trim()}.${file.split('.').pop()}`;
if(name ! == file) { renameSync(fullPath, join(rootDir, name)); } }).catch((err) => {if (statSync(fullPath).size < 10) {
unlinkSync(fullPath);
} else {
console.log('parse error'.statSync(fullPath).size); console.error(err.message); }}); }); readdirSync(rootDir).filter((file) => {if (file.includes(The '-')) {
let lastPart = file.split(The '-') [1]; const name = lastPart.replace(/\.. + /,' ');
const target = join(rootDir, name);
if (!existsSync(target)) mkdirSync(target);
renameSync(join(rootDir, file), join(target, file));
}
});
Copy the codeother
There are a lot of things you can do with Node in just a dozen lines of code, like before:
- I worked as an intelligent device gateway for the company, monitored whether the door of the company forgot to close, and quickly notified the person in charge through the wechat interface of the company.
- Write a simple UDP script to control the automatic sleep and wake up of various devices in the home.
- Some game save modifier XD
- Quickly crawl any websites you visit.
I now have a small toss group, which gathered some like to toss small partners.
In the case of no advertisement, we will talk about software, HomeLab and some programming problems together, and also share some technical salon information in the group from time to time.
Like to toss small partners welcome to scan code to add friends. (Please indicate source and purpose, otherwise it will not be approved.
All this stuff about getting into groups