origin
When installing project dependencies, the terminal reported the following error, resulting in the dependency installation failure.
Through the error message, we can see the reason for the @sentry/cli package. Since the project does not rely on this package directly, in order to eliminate the influence between packages, we created a new folder and installed this package separately, but found the same error. Then I asked my colleague to install the package and try it, and found that everything was normal and there was no error.
The following is a series of operations: Google search, Github issue search, change to NPM installation, switch NPM source, switch node version, install another version @sentry/ CLI, clear yarn and NPM cache, restart the computer… But discovery doesn’t help…
It seems that things are not so simple
Error: Nodescripts /install.js was executed when the error was detected. Clone @sentry/cli to nodescripts /install.js
Found that actually perform/Users/sliwey githome/sentry – cli/sentry – cli – the error occurred while version command, according to the above path found much in the project root directory called sentry – cli executable file.
If you look at the scripts/install.js code, you’ll see that it does exactly the same thing:
downloadBinary()
.then((a)= > checkVersion())
.then((a)= > process.exit(0))
.catch(e= > {
console.error(e.toString());
process.exit(1);
});
Copy the codeJust download an executable file and check the version number. DownloadBinary (I’ve simplified the code and added a few comments, and you can see the code at github.com/getsentry/s…) :
function downloadBinary() {
const arch = os.arch();
const platform = os.platform();
const outputPath = helper.getPath();
// Get the download link according to the system
const downloadUrl = getDownloadUrl(platform, arch);
// Generate the cache path based on the download link
const cachedPath = getCachedPath(downloadUrl);
// If a cache hit, the file is copied to the current path
if (fs.existsSync(cachedPath)) {
copyFileSync(cachedPath, outputPath);
return Promise.resolve();
}
// If the cache is not hit, download the file and write it to the cache
return fetch(downloadUrl, { redirect: 'follow', agent }).then(response= > {
const tempPath = getTempFile(cachedPath);
mkdirp.sync(path.dirname(tempPath));
return new Promise((resolve, reject) = > {
response.body
.pipe(fs.createWriteStream(tempPath, { mode: '0755' }))
}).then((a)= > {
copyFileSync(tempPath, cachedPath);
copyFileSync(tempPath, outputPath);
fs.unlinkSync(tempPath);
});
});
}
Copy the codeThe executable file was downloaded from the cache. The cache file was not downloaded from the cache.
Based on the obtained path, delete the corresponding files and reinstall, everything is OK ~
Here’s the point
The problem is resolved, but recall the previous operation, which did cache clearing, including YARN and NPM, using the following two commands:
yarn cache clean
npm cache clean --force
Copy the codeThe sentry-cli cache is stored in ~/. NPM. Therefore, it has nothing to do with YARN. NPM: NPM cache clean –force: NPM cache clean –force: NPM ~/.npm Npm-cache = nPm-cache
To make it easier to read, I’ve cut out a few images:
At first glance, it seems that there is nothing wrong with the cache configuration.
lib/cache.js
function clean (args) {
if(! args) args = []if (args.length) {
return BB.reject(new Error('npm cache clear does not accept arguments'))}// That's the point
// npm.cache is ~/.npm
// So cachePath should be ~/.npm/_cacache
const cachePath = path.join(npm.cache, '_cacache')
if(! npm.config.get('force')) {
return BB.reject(new Error("As of npm@5, the npm cache self-heals from corruption issues and data extracted from the cache is guaranteed to be valid. If you want to make sure everything is consistent, use 'npm cache verify' instead. On the other hand, if you're debugging an issue with the installer, you can use `npm install --cache /tmp/empty-cache` to use a temporary cache instead of nuking the actual one.\n\nIf you're sure you want to delete the entire cache, rerun this command with --force."))}// TODO - remove specific packages or package versions
return rm(cachePath)
}
Copy the codeNPM cache clean –force (~/. NPM /_cacache);
On second thought, this should not have been mentioned in the document, but when I went back to the document, I found that I had missed a point…
As follows:
Simply put, after npm@5, NPM places the cache data in the _cacache folder below the path configured for the cache field in the configuration file. Based on the contents of the above two paragraphs, it can be concluded that:
- In the configuration file
cacheField configures the root directory - Cached data is placed in the root directory
_cacachefolder cleanThe command clears_cacachefolder
What exactly does the NPM cache hold
When you open the _cacache folder, instead of having multiple packages like node_modules, you’ll find something like this:
If you open content-v2, you’ll find that it’s basically a bunch of binaries. Change the extension of the binaries to.tgz and unzip it, and you’ll find that it’s in the familiar NPM package. Index-v5 contains some descriptive files, which are also the indexes of the files in Content-v2. If you look closely, you can see that they look like HTTP response headers, with cache-specific values:
So how are these files generated? As you can see from the above documentation, NPM mainly uses Pacote to install packages. Let’s take a look at how NPM uses Pacote in code. NPM uses PacoTE in the following three areas:
- NPM install XXX (yes
pacote.extractUnzip the corresponding package in the correspondingnode_modulesThe following. NPM source code entrylib/install/action/extract-worker.js, pacote source code entry:extract.js) - NPM cache add XXX (pass
pacote.tarball.stream往~/.npm/_cacacheAdd cached data in. NPM source code entrylib/cache.js, pacote source code entry:tarball.js#tarballStream) - NPM Pack XXX (passed
pacote.tarball.toFileThe compressed file is generated in the current path. NPM source code entrylib/pack.js, pacote source code entry:tarball.js#tarballToFile)
Comparing the above three pacote methods, it can be found that the main method it relies on is lib/ withtarballstream.js, which has a lot of code, so it can be simplified by mainly looking at the Chinese notes:
function withTarballStream (spec, opts, streamHandler) {
opts = optCheck(opts)
spec = npa(spec, opts.where)
// Read the local file
consttryFile = ( ! opts.preferOnline && opts.integrity && opts.resolved && opts.resolved.startsWith('file:'))? BB.try((a)= > {
const file = path.resolve(opts.where || '. ', opts.resolved.substr(5))
return statAsync(file)
.then((a)= > {
const verifier = ssri.integrityStream({ integrity: opts.integrity })
const stream = fs.createReadStream(file)
.on('error', err => verifier.emit('error', err))
.pipe(verifier)
return streamHandler(stream)
})
: BB.reject(Object.assign(new Error('no file! '), { code: 'ENOENT' }))
After the previous reject step, read from the cache
const tryDigest = tryFile
.catch(err= > {
if( opts.preferOnline || ! opts.cache || ! opts.integrity || ! RETRIABLE_ERRORS.has(err.code) ) {throw err
} else {
// Use cacache to read data from the cache
const stream = cacache.get.stream.byDigest(
opts.cache, opts.integrity, opts
)
stream.once('error', err => stream.on('newListener', (ev, l) => {
if (ev === 'error') { l(err) }
}))
return streamHandler(stream)
.catch(err= > {
if (err.code === 'EINTEGRITY' || err.code === 'Z_DATA_ERROR') {
opts.log.warn('tarball'.`cached data for ${spec} (${opts.integrity}) seems to be corrupted. Refreshing cache.`)
If the error code is EINTEGRITY or Z_DATA_ERROR, the cache is cleared
return cleanUpCached(opts.cache, opts.integrity, opts)
.then((a)= > { throw err })
} else {
throw err
}
})
}
})
// Reject before downloading
const trySpec = tryDigest
.catch(err= > {
if(! RETRIABLE_ERRORS.has(err.code)) {// If it's not one of our retriable errors, bail out and give up.
throw err
} else {
return BB.resolve(retry((tryAgain, attemptNum) = > {
// Download the package, which is actually downloaded via npm-registry-fetch
const tardata = fetch.tarball(spec, opts)
if(! opts.resolved) { tardata.on('manifest', m => {
opts = opts.concat({ resolved: m._resolved })
})
tardata.on('integrity', i => {
opts = opts.concat({ integrity: i })
})
}
return BB.try((a)= > streamHandler(tardata))
}, { retries: 1}}})))return trySpec
.catch(err= > {
if (err.code === 'EINTEGRITY') {
err.message = `Verification failed while extracting ${spec}:\n${err.message}`
}
throw err
})
}
Copy the codeFrom the above code, you can see that Pacote relies on nPm-Registry-fetch to download packages. A look at the npm-registry-fetch documentation shows that there is a cache attribute that can be set at request: npm-registry-fetch#opts.cache
If the cache value is set (~/.npm/_cacache in NPM), the cache data generated according to IETF RFC 7234 will be created in the given path. Open the RFC address and find the document that describes the HTTP cache, so the index-v5 file at the beginning of this paragraph should make sense.
A quick summary:
~/.npm/_cacacheStorage is some binary files, and the corresponding index.- NPM install, if there is a cache, will use Pacote to decompress the corresponding binary file to the corresponding
node_modulesThe following. - NPM itself only provides a method to clear the cache and verify the integrity of the cache, does not provide a method to directly manipulate the cache, can pass
cacacheTo manipulate the cached data.
Write in the last
Reviewing the whole thing, I realized how important it is to look carefully at the documentation! Keep in mind! Keep in mind! But also usually do not pay attention to the point combed again, also can be a harvest, in the form of text to record, easy to review.
Original link: github.com/sliwey/blog…