Read one NPM module (2) – MEM every day

Series of articles:

1. Read one NPM module (1) – username every day

Sindre Sorhus (@sindre Sorhus, @sindre Sorhus, @sindre Sorhus, @sindre Sorhus, @sindre Sorhus) @Sindre Sorhus (@sindre) @Sindre Sorhus (@sindre) @Sindre Sorhus (@sindre) @Sindre Upgrade to version 4.0.0, but the core code has not changed much 😊

A one-sentence introduction

The NPM module read today is MEM, which improves performance by caching the return value of a function to reduce the actual number of times the function is executed. The current version is 3.0.1 and has about 3.5 million weekly downloads.

usage

const mem = require('mem');
 
// Synchronize function cache
let i = 0;
const counter = (a)= > ++i;
const memoized = mem(counter);
 
memoized('foo');
/ / = > 1
 
memoized('foo');
//=> 1 If the parameters are the same, return 1
 
memoized('bar');
//=> 2 parameter changes, the counter function is executed again, return 2
 
memoized('bar');
/ / = > 2

// Asynchronous function cache
let j = 0;
const asyncCounter = (a)= > Promise.resolve(++j);
const asyncmemoized = mem(asyncCounter);

asyncmemoized().then(a= > {
    console.log(a);
    / / = > 1
 
    asyncmemoized().then(b= > {
        console.log(b);
        / / = > 1
    });
});
Copy the code

The above usage is the core function of MEM. Besides, it also supports setting cache time, customizing cache Hash value, and counting cache hit data.

The source code to learn

The hash function

In order for the meM function to return the same value for the same argument, it must hash the argument, and then cache the hash result as key and the function result as value. Here’s a simple example:

const cache = {};

// Cache the run results of arg1
const key1 = getHash(arg1);
cache[key1] = func(arg1);

// Cache the run results of arg2
const key2 = getHash(arg2);
cache[key2] = func(arg2);
Copy the code

The key is the getHash hash function: How do you handle different data types? How are comparisons between objects handled? This is a common interview question: How do you compare? Take a look at the source code:

// Source code 2-1: mem hash function
const defaultCacheKey = (. args) = > {
	if (args.length === 1) {
		const [firstArgument] = args;
		if (
			firstArgument === null ||
			firstArgument === undefined| | -typeoffirstArgument ! = ='function' && typeoffirstArgument ! = ='object')) {returnfirstArgument; }}return JSON.stringify(args);
};
Copy the code

As you can see from the code above:

1. When only one parameter, and the parameter is null | undefined or type not for function | object, hash function parameters directly to return.
2. If this is not the case, the parameter is returnedJSON.stringify()The value of the.

First review the ES6 can defines the data type, which includes 6 kinds of primitive types (Boolean | Nunber | Null | Undefined | String | Symbol) and Object types. The hash function in the source code needs to distinguish between different types because a direct comparison of the Object type does not match what we want to achieve here:

const object1 = {a: 1};
const object2 = {a: 1};

console.log(object1 === object2);
// => flase

// Expected effect
console.log(defaultCacheKey(object1) === defaultCacheKey(object2));
// => true
Copy the code

At first I thought that the authors would be using different types of data to make specific processing (similar to Lodash’s _.isequal () implementation), but the more violent method is to convert Object data directly to a string via json.stringify (). When I first saw it, I was shocked — I’d only heard people do it as a joke, but I didn’t think it would happen.

This approach is simple and highly readable, but there are problems:

1. Json.stringify () can take a lot of time when the object structure is complex.

2. Json.stringify () results in {} for different re objects, which is not what a hash function would expect.

   console.log(JSON.stringify(/Sindre Sorhus/));
   / / = > '{}'
   
   console.log(JSON.stringify(/Elvin Peng/));
   / / = > '{}'
   Copy the code

The first problem is good, because meM supports passing in custom hash functions if performance is a problem with json.stringify () hashing, which can be solved by writing your own efficient hash functions.

The second problem is that the function function does not meet expectations and needs to be bugfixed.

Storage structure

Without additional arguments, the support source code for synchronization functions can be simplified as follows:

// source code 2-2 MEM core logic
const mimicFn = require('mimic-fn');

const cacheStore = new WeakMap(a);module.exports = (fn) = > {
    const memoized = function (. args) {
        const cache = cacheStore.get(memoized);
        constkey = defaultCacheKey(... args);if (cache.has(key)) {
            const c = cache.get(key);
            return c.data;
        }

        const ret = fn.call(this. args);const setData = (key, data) = > {
            cache.set(key, {
                data,
            });
        };
        
        setData(key, ret);
        
        return ret;
    }
    
    const retCache = new Map(a); mimicFn(memoized, fn); cacheStore.set(memoized, retCache);return memoized;
}

Copy the code

The overall logic is very clear, mainly to complete two actions:

1. Will be of typeMap 的 retCacheAs a cache of the result of function execution, the key value of the cache isdefaultCacheKeyThe result of the hash.
2. Will be of typeWeakMap 的 cacheStoreAs a cache as a whole, the key value of the cache is the function itself.

The second level cache formed by the above two actions realizes the core functions of the module, and the choice of the two types is worth exploring.

RetCache selects Map instead of Object mainly because Map key values support all types, whereas Object key values support only strings. In addition, Map is preferred for cache data structures in the following ways:

• Map.sizeProperty to conveniently obtain the current number of caches
• MapType of supportclear() | forEach()And other commonly used tool functions
• MapTypes are iterable by default, that is, supportediterable protocol

The main reason why cacheStore chooses WeakMap over Map is that it does not increase the number of references and is better for node.js garbage collection.

Asynchronous support

I was going to write about asynchronous support, but now that it’s 1am, I’d rather forget it and go to bed early 😪

Interested friends can read by themselves ~

Write in the last

In addition to the one Bug mentioned above, MEM also has the possibility of memory leaks: The cached data will not be automatically cleared after it has expired (i.e., it has been cached for longer than the specified maxAge). This may result in a Memory leak caused by the inability to release the Memory occupied by the invalid cache when too much data is cached. Old results are not deleted from the cache.

MimicFn (memoized, FN) was intentionally omitted from the reading of source code 2-2; Why? Because I will read the module mimicFn tomorrow, I hope you can continue to support me.

About me: graduated from huake, working in Tencent, elvin’s blog welcome to visit ^_^