Reading Note: There is no reason to insist so strongly that your chosen technology is the best, and to belittle or even ignore other technologies. If you insist, you will end up hurting yourself. If you’re willing to keep an open mind about technology, rather than sticking with what you already know and claiming it’s the best, you’ll find more opportunities open up for you. ~ ~ ~

A memory leak

Node is very sensitive to memory leakage. With tens of thousands of traffic, a byte leakage will cause accumulation. In garbage collection, scanning objects will take more time, affecting the response time of the entire application, until the process memory overflow and application crash.

Causes of memory overflow:

  • The cache
  • Queue consumption is not timely
  • Scope is not released

Cache (memory is only used as cache)

  • Caching plays an important role in applications and can save resources effectively. If there is a cache hit, you can save one I/O
  • If an object is used as a cache in Node, it will reside in the old generation. If there are more keys in the cache, there will be more long-lived objects, making them useless for garbage collection. Affects the entire application.
  • JS developers like to use the keys of an object to cache things, which is strictly different from caching. Strict caching has an expiration policy, while objects do not. Because of the garbage collection mechanism, objects can only be used sparingly in the cache.
  1. Cache restriction policy

To solve the problem that objects in the cache cannot be freed, a policy is needed to limit the unlimited growth of the cache

    var LimitableMap = function (limit) { 
     this.limit = limit| | 10; // Limit the size of this.map = {}; this.keys = []; }; var hasOwnProperty = Object.prototype.hasOwnProperty; LimitableMap.prototype.set =function (key, value) { 
     var map = this.map; 
     var keys = this.keys;
     if(! hasOwnProperty.call(map, key)) {if(keys.length === this.limit) { var firstKey = keys.shift(); Delete map[firstKey]; } keys.push(key); } map[key] = value; }; LimitableMap.prototype.get =function (key) { 
     return this.map[key]; 
    }; 
module.exports = LimitableMap;
Copy the code
  • Module mechanism causes memory leak, resolve policy

Module introduction. To speed up module introduction, all modules are compiled and cached. Functions exported by exports can access private variables in file modules, so that the scope generated by file module compilation is not released because of module cache. Because of the caching mechanism of modules, modules are resident in the old generation. (Be careful of memory leaks)

    var leakArray = []; 
    exports.leak = function () { 
        leakArray.push("leak"+ Math.random()); }; <! -- Every time an external leakArray is called, leakArray keeps adding to leakArray, taking up memory and not being released. -->Copy the code

If the module inevitably uses the above design, add an interface to clear the queue.

  1. Caching solutions

Be careful about using memory directly as a cache, and there are many problems if you’re not careful

  • Limit the size of the cache, otherwise it will have a significant impact on garbage collection and memory space
  • Unable to contribute memory information between processes (cached information)
  • Each process has a cache, and the use of physical memory is a waste.
  • The solution
    • If you use a lot of caching, a good solution is to use an out-of-process cache that does not cache state itself. External caches have good cache expiration policies and memory management without affecting node process performance.
    • Redis and Memcached

Pay attention to the state of the queue

After addressing memory leaks from caches, another area that can inadvertently cause memory leaks is queues

  • JS can fulfill special requirements through queues (array objects), which often serve as intermediates in consumer-generators.
  • In most cases, the rate of consumption is much faster than the rate of generation, and memory is not easily leaked. But once the consumption rate is less than the generation rate, it will cause accumulation.
  • For example: write log, if use database to store log, log is usually massive, database is built on the file system, write efficiency is far lower than file write, so the formation of database write accumulation, Js scope will not be released, memory occupation will not fall, causing memory leakage.
  • Solution: 1. Write to a file, but there is also a memory leak (generation speed for some reason, or consumption for some reason, sharp drop). 2. Monitor the length of the queue and alert the developers if there is a build-up. Or add timeouts to asynchronous calls. Pass a timeout exception when a timeout occurs.