MMKV principle article

1. Memory preparation

The MMAP memory mapping file provides a memory block that can be written at any time. App only writes data into it, and the operating system is responsible for writing the memory back to the file. There is no need to worry about data loss caused by crash.

Mmap can be used in two ways. One is to create an anonymous mapping, which can share memory between parent and child processes. The other is the virtual address space of the disk file mapping process. MMKV is the disk file mapping used.

Common read/write data: user space —- kernel space — disk

MMAP: user space —- disk

2. Data organization

With respect to data serialization, we use protobuf protocol, and PB has a good performance in performance and space occupancy. Considering that we are providing a general KV component, key can be limited to string and value can be varied (int/bool/double, etc.). To be generic, consider serializing values into a unified buffer using the Protobuf protocol, which can then serialize these KV objects into memory.

Third, write optimization

Standard Protobuf does not provide incremental update capability; each write must be written in full. Considering that the main usage scenario is frequent write update, we need the ability of incremental update: append the incremental KV object directly to the end of memory after serialization; In this way, there will be several new and old copies of the same key, with the latest data at the end. Therefore, when the program starts and turns on MMKV for the first time, the data can be guaranteed to be up-to-date and effective by constantly replacing the previous value with the value read later

4. Spatial optimization

Using Append for incremental updates brings up a new problem, which is that the file size can grow out of control if you append constantly. For example, if the same key is constantly updated, it may consume hundreds of M or even G space. In fact, the entire KV file is only one key, which can be saved in less than 1K space. This is clearly undesirable. We need to make a compromise between performance and space: apply space in unit of memory pagesize, append mode until space runs out; When append to the end of the file, file reorganization, key rearrangement, try serialization save the rearrangement result; If you don’t have enough space after reloading, double the size of the file until you have enough space.

  • The timing of data rewrite
    • File free space is less than the new key-value pair size
    • Hash to empty
  • File Expansion Opportunity
    • The required space is 1.5 times larger than the total size of the current file, and is expanded to s times the previous size

5. Data validity

CRC check and length check and recovery mechanism are provided, but data recovery is often unreliable.

Six, encrypted storage

MMKV uses AES CFB-128 algorithm to encrypt/decrypt. We choose CFB instead of the common CBC algorithm, mainly because MMKV uses append-only to implement insert/update operation and streaming encryption algorithm is more suitable

7. Multiple processes

1.ContentProvider

(1) Separate process management data, data synchronization is not easy to make mistakes, simple and easy to use

(2) Mmap files to the memory space of each accessing process

2. Process lock File lock

(1) Recursive locking: if a process/thread already has a lock, subsequent locking operations will not cause the lock to be locked, and unlocking will not cause the outer lock to be released. For file locks, the former is fine, but the latter is not. Because a file lock is a state lock, there is no counter, and no matter how many times the lock is added, a single unlock operation is resolved. Recursive locking is very much needed whenever you use subfunctions.

(2) Lock upgrade/downgrade Lock Upgrade refers to upgrading a shared lock to a mutex lock, that is, upgrading a read lock to a write lock. Lock degradation works the other way around. File locks support lock upgrades, but are prone to deadlocks: if A and B both hold read locks and want to upgrade to write locks, they will be stuck waiting for each other and deadlock will occur. In addition, because the file lock does not support recursive locking, it also leads to lock degradation cannot be carried out, once dropped to no lock

  • If the read lock is already held, try adding the write lock first. The try_lock failure indicates that another process has the read lock. You need to release your own read lock before adding the write lock to avoid deadlocks.
  • When unlocking a write lock, we cannot directly release the write lock if we previously held the read lock, which will cause the read lock to be unlocked. We should degrade the lock by adding a read lock.
  1. Write pointer synchronization We can cache our own write pointer within each process, and then write the latest write pointer position to MMAP memory as well as the key value. In this way, each process only needs to compare the cached pointer to the mMAP memory write pointer. If the pointer is different, it indicates that another process wrote. In fact, MMKV already stores the effective memory size in the file header. This value is exactly the memory offset of the write pointer. We can reuse this value to check the write pointer.
  2. The perception of memory refactoring considers using a monotonically increasing sequence number, incrementing the sequence number each time a memory refactoring occurs. This sequence number is also placed in MMAP memory, and a copy is cached internally for each process. By comparing the sequence number, you can know whether other processes triggered memory reordering.
  3. Perception of memory growth In fact, MMKV will try to free up memory space through memory refactoring before memory growth, and then apply for new memory after memory refactoring is insufficient. So memory growth can be treated like memory refactoring. The new memory size can be obtained by querying the file size without having to store it separately in MMAP memory.

Variable-length coding principle for Protobuf

Principle:

1. Variable length encoding & skipping optional fields

2. Applies to the network transmission process

First, storage type

TAG_[LENGTH]_VALUE

TAG: filedId(top 5)+ WIRE_TYPE(bottom 3)

LENGTH: exists when WIRE_TYPE = 2

VALUE:WIRE_TYPE = varint Specifies the small – end storage mode. Other values can be read normally

Second, the WIRE_TYPE

0: varINT variable length encoding, mainly depends on this to reduce the storage volume

1:8 bytes in fixed length

2: specifies the length

3, 4: Obsolete

5:4 bytes in fixed length

Varint principle

  1. The number of type INT32 generally requires four bytes. But with Varint, small int32 numbers can be represented by 1 byte.

  2. Using Varint notation, large numbers are represented in 5 bytes. From a statistical point of view, not all numbers in a message are large, so in most cases, with Varint, numeric information can be represented in fewer bytes

  3. Small end storage mode

    Example 1:

    For the number 1, the binary is: 00000000 00000000 00000000 00000001

    PB can store this value in a single byte: 0 00000001

    The first 0 indicates that the byte is the end byte, and the last seven bits 0000001 indicate the decimal number 1

    Example 2:

    For the number 500, the binary is: 00000000 00000000 00000001 11110100

The seven-digit split from the lowest digit is: 1110100 0000011

The PB code is represented by two bytes: 1 1110100 0 0000011

Decoder: A high value of 1 indicates that one byte is to be read

Remove the highest bit: 1110100 0000011

Because it is small endian mode: 00000111110100 is 500 in decimal

Iii. ZigTag coding (solve the problem of occupying multiple bytes for negative numbers)

  • Source code: the highest bit is a sign bit, the rest of the absolute value;

  • Reverse code: in addition to the symbol bits, the remaining bits of the original code are reversed successively;

  • Complement: For positive numbers, the complement is itself; For negative numbers, the remaining bits of the source code are reversed and then +1, except for the sign bits

    Defects in source code:

    1, 0 has two manifestations: 00000000 and 10000000

    1 + (-1) = 00000001 + 10000001 = 10000010 = -2

1+ (-1) = 00000001+ 11111111 = 00000000 = 0

n hex h(n) ZigZag (hex)
0 00 00 00 00 00 00 00 00 00
– 1 ff ff ff ff 00 00 00 01 01
1 00 00 00 01 00 00 00 02 02
2 – ff ff ff fe 00 00 00 ’03 03
2 00 00 00 02 00 00 00 04 04
. . . .
– 64. ff ff ff c0 00 00 00 7f 7f
64 00 00 00 40 00 00 00 to 80 80 01
. . . .

After receiving the hash value, the assumed encoding strategy is to remove the byte after the hash value leading 0 as the compression encoding.

Fourth, the sample

Example 1:

message Test1 {

optional int32 a = 1; // fildId is 1

}

Create the Test1 message and set a to 150

Code: 08 96 01

Decode: 00001000 10010110 00000001

(1) 000001000:000 indicates that WIRE_TYPE = 0, that is, Varint; 00001 = 1 corresponds to the first fieldId;

(2) 10010110 00000001:

10010110 the highest bit 1 indicates that the next byte needs to be read, and 0010110 is left.

00000001 The highest bit indicates that the next byte is not read. 0000001 remains.

00000010010110 is converted to base 10, that is, 150

Example 2: Embedded message

Message Test2{

​ optional Test1 c = 3;

}

Example 1: assign 150 to test1a

The resulting code: 1A 03 08 96 01

The last three digits of 1A correspond to base 2:00011010. The first five digits of 3 correspond to fieldId

03 indicates length: and the length of 3 bytes

08 96 01: See example 1 analysis process

MMKV source code analysis (Java layer)

Member variables

// Error recovery policy related
private static final EnumMap<MMKVRecoverStrategic, Integer> recoverIndex = new EnumMap(MMKVRecoverStrategic.class);
//MMKV log output is related
private static final EnumMap<MMKVLogLevel, Integer> logLevel2Index;
private static final MMKVLogLevel[] index2LogLevel;
// Store the set of open MMKV descriptors
private static final Set<Long> checkedHandleSet;
//MMKV file root directory
private static String rootDir;
//MMKV process mode
public static final int SINGLE_PROCESS_MODE = 1;
public static final int MULTI_PROCESS_MODE = 2;
private static final int CONTEXT_MODE_MULTI_PROCESS = 4;
private static final int ASHMEM_MODE = 8;
// Serialize related
private static finalHashMap<String, Creator<? >> mCreators;// Handle log redirection and file error recovery policies
private static MMKVHandler gCallbackHandler;
private static boolean gWantLogReDirecting;
// Content change notifications are called back when updated by other processes
private static MMKVContentChangeNotification gContentChangeNotify;
// the MMKV descriptor is used to read and write files
private final long nativeHandle;
Copy the code

Two, API interface

1. Initialize the MMKV

// Pass in the context initialization function, log default is LevelInfo
public static String initialize(Context context) {
    String root = context.getFilesDir().getAbsolutePath() + "/mmkv";
    MMKVLogLevel logLevel = MMKVLogLevel.LevelInfo;
    return initialize((String)root, (MMKV.LibLoader)null, logLevel);
}
// Pass in context and logLevel
public static String initialize(Context context, MMKVLogLevel logLevel) {
    String root = context.getFilesDir().getAbsolutePath() + "/mmkv";
    return initialize((String)root, (MMKV.LibLoader)null, logLevel);
}
// Pass in the context and loader
public static String initialize(Context context, MMKV.LibLoader loader) {
    String root = context.getFilesDir().getAbsolutePath() + "/mmkv";
    MMKVLogLevel logLevel = MMKVLogLevel.LevelInfo;
    return initialize(root, loader, logLevel);
}
// Pass context, loader/level
public static String initialize(Context context, MMKV.LibLoader loader, MMKVLogLevel logLevel) {
    String root = context.getFilesDir().getAbsolutePath() + "/mmkv";
    return initialize(root, loader, logLevel);
}
// The incoming path
public static String initialize(String rootDir) {
    MMKVLogLevel logLevel = MMKVLogLevel.LevelInfo;
    return initialize((String)rootDir, (MMKV.LibLoader)null, logLevel);
}
// The incoming path and log level
public static String initialize(String rootDir, MMKVLogLevel logLevel) {
    return initialize((String)rootDir, (MMKV.LibLoader)null, logLevel);
}
// Incoming path and loader
public static String initialize(String rootDir, MMKV.LibLoader loader) {
    MMKVLogLevel logLevel = MMKVLogLevel.LevelInfo;
    return initialize(rootDir, loader, logLevel);
}
// This method is finally called in the path /loader and log level
public static String initialize(String rootDir, MMKV.LibLoader loader, MMKVLogLevel logLevel) {
    // if the loader is not empty, the c++_shared and MMKV libraries will be loaded by the incoming loader, otherwise the System. LoadLibrary will be loaded
    if(loader ! =null) {
        if ("SharedCpp".equals("SharedCpp")) {
            loader.loadLibrary("c++_shared");
        }

        loader.loadLibrary("mmkv");
    } else {
        if ("SharedCpp".equals("SharedCpp")) {
            System.loadLibrary("c++_shared");
        }

        System.loadLibrary("mmkv");
    }
    // jNI calls native methods
    jniInitialize(rootDir, logLevel2Int(logLevel));
    / / record rootDir
    MMKV.rootDir = rootDir;
    returnMMKV.rootDir; Initialize (String rootDir, mmkv.libloader loader, MMKVLogLevel logLevel)Copy the code

2. Obtain the MMKV storage root path

// You can use this method to determine whether the path is empty or initialized successfully
public static String getRootDir(a) {
    return rootDir;
}

Copy the code

3. Obtain MMKV

// The path is obtained by mapId. By default, the path must be initialized, that is, the path is not empty
@Nullable
public static MMKV mmkvWithID(String mmapID) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getMMKVWithID(mmapID, 1, (String)null, (String)null);
        return checkProcessMode(handle, mmapID, 1); }}// Obtain the value from mapId and process mode
@Nullable
public static MMKV mmkvWithID(String mmapID, int mode) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getMMKVWithID(mmapID, mode, (String)null, (String)null);
        returncheckProcessMode(handle, mmapID, mode); }}// Obtain the value by using mapId mode and key
@Nullable
public static MMKV mmkvWithID(String mmapID, int mode, @Nullable String cryptKey) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getMMKVWithID(mmapID, mode, cryptKey, (String)null);
        returncheckProcessMode(handle, mmapID, mode); }}// Obtain the value from mapID and rootPath
@Nullable
public static MMKV mmkvWithID(String mmapID, String rootPath) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getMMKVWithID(mmapID, 1, (String)null, rootPath);
        return checkProcessMode(handle, mmapID, 1); }}// Obtain the value from the mapId rootPath process and key
@Nullable
public static MMKV mmkvWithID(String mmapID, int mode, @Nullable String cryptKey, String rootPath) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getMMKVWithID(mmapID, mode, cryptKey, rootPath);
        returncheckProcessMode(handle, mmapID, mode); }}// Obtain handle /size/ mode/key corresponding to cross-process TRANSMISSION of MMKV in anonymous shared memory mode
@Nullable
public static MMKV mmkvWithAshmemID(Context context, String mmapID, int size, int mode, @Nullable String cryptKey) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        String processName = MMKVContentProvider.getProcessNameByPID(context, Process.myPid());
        if(processName ! =null&& processName.length() ! =0) {
            if (processName.contains(":")) {
                Uri uri = MMKVContentProvider.contentUri(context);
                if (uri == null) {
                    simpleLog(MMKVLogLevel.LevelError, "MMKVContentProvider has invalid authority");
                    return null;
                } else {
                    simpleLog(MMKVLogLevel.LevelInfo, "getting parcelable mmkv in process, Uri = " + uri);
                    Bundle extras = new Bundle();
                    extras.putInt("KEY_SIZE", size);
                    extras.putInt("KEY_MODE", mode);
                    if(cryptKey ! =null) {
                        extras.putString("KEY_CRYPT", cryptKey);
                    }

                    ContentResolver resolver = context.getContentResolver();
                    Bundle result = resolver.call(uri, "mmkvFromAshmemID", mmapID, extras);
                    if(result ! =null) {
                        result.setClassLoader(ParcelableMMKV.class.getClassLoader());
                        ParcelableMMKV parcelableMMKV = (ParcelableMMKV)result.getParcelable("KEY");
                        if(parcelableMMKV ! =null) {
                            MMKV mmkv = parcelableMMKV.toMMKV();
                            if(mmkv ! =null) {
                                simpleLog(MMKVLogLevel.LevelInfo, mmkv.mmapID() + " fd = " + mmkv.ashmemFD() + ", meta fd = " + mmkv.ashmemMetaFD());
                            }

                            returnmmkv; }}return null; }}else {
                simpleLog(MMKVLogLevel.LevelInfo, "getting mmkv in main process");
                mode |= 8;
                long handle = getMMKVWithIDAndSize(mmapID, size, mode, cryptKey);
                return newMMKV(handle); }}else {
            simpleLog(MMKVLogLevel.LevelError, "process name detect fail, try again later");
            return null; }}}// Get the default storage MMKV
@Nullable
public static MMKV defaultMMKV(a) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        // The default mode is single-process, no encryption, and the default mapId is DefaultMMKV
        long handle = getDefaultMMKV(1, (String)null);
        return checkProcessMode(handle, "DefaultMMKV".1); }}// Get the default storage file
@Nullable
public static MMKV defaultMMKV(int mode, @Nullable String cryptKey) {
    if (rootDir == null) {
        throw new IllegalStateException("You should Call MMKV.initialize() first.");
    } else {
        long handle = getDefaultMMKV(mode, cryptKey);
        return checkProcessMode(handle, "DefaultMMKV", mode); }}// Check the process mode handle similar handle cannot be 0. Check the process mode. If the process mode does not match, an exception will be thrown
// Handle will store it to the local set. If there is no local record of Handle, the current mode will be checked against Handle. If there is no match, an error will be reported and an exception will be thrown
Private final Long nativeHandle, i.e. New MMKV();
// Please refer to the analysis in the following chapters to find that if different process modes are used to obtain the same file name, the handle value obtained is the same and no error is reported
@Nullable
private static MMKV checkProcessMode(long handle, String mmapID, int mode) {
    if (handle == 0L) {
        return null;
    } else {
        if(! checkedHandleSet.contains(handle)) {if(! checkProcessMode(handle)) { String message;if (mode == 1) {
                    message = "Opening a multi-process MMKV instance [" + mmapID + "] with SINGLE_PROCESS_MODE!";
                } else {
                    message = "Opening a MMKV instance [" + mmapID + "] with MULTI_PROCESS_MODE, ";
                    message = message + "while it's already been opened with SINGLE_PROCESS_MODE by someone somewhere else!";
                }

                throw new IllegalArgumentException(message);
            }

            checkedHandleSet.add(handle);
        }

        return newMMKV(handle); }}Copy the code

4. Functional interfaces

/**** Exit function ******/

// Exit the MMKV, for the global MMKV, you must call initialize again after exit, otherwise the MMKV instance will throw an exception
public static native void onExit(a);
// When an MMKV instance is no longer in use, you can call this method to shut it down and obtain the instance again later
public native void close(a);

/*** * Operation encryption and decryption related functions ***/

// Obtain the secret key for a single MMKV instance
@Nullable
public native String cryptKey(a);
// Reset the key, for a single MMKV file, return reset success or failure
public native boolean reKey(@Nullable String var1);
/ / for validation
public native void checkReSetCryptKey(@Nullable String var1);

/***** Obtain size-related functions ****/

// Static methods get pagesize returns the default value 4K
public static native int pageSize(a);
// To obtain the size of the value corresponding to a key, PB encoding is required, for example, integer 1, PB encoding is used to obtain 1,500, and 2 is obtained. For reference types such as string, since PB encoding inserts bytes to indicate length, the size obtained here contains length bytes, such as "12345". So here we get 6
public int getValueSize(String key) {
    return this.valueSize(this.nativeHandle, key, false);
}
In most cases, the obtained value is the same as the above getValueSize value. However, for string types such as "12345", the obtained value is 5, and the bytes representing the length are removed
public int getValueActualSize(String key) {
    return this.valueSize(this.nativeHandle, key, true);
}
// Returns a multiple of pagesize
public long totalSize(a) {
    return this.totalSize(this.nativeHandle);
}
// The test result is the number of keys (excluding duplicates); RemoveValueForkey will also remove the key, which will be 0 after clearALL
public long count(a) {
    return this.count(this.nativeHandle);
}

/*** Operation key related functions ***/

// Whether to contain a key
public boolean containsKey(String key) {
    return this.containsKey(this.nativeHandle, key);
}
// Delete the value of a key, and the key will also be deleted
public void removeValueForKey(String key) {
    this.removeValueForKey(this.nativeHandle, key);
}
@Nullable
// Get all keys
public native String[] allKeys();
// Delete the value of the related key
public native void removeValuesForKeys(String[] var1);

/*** * clear function ***/
// Clear all data for a single MMKV
public native void clearAll(a);
// Trigger alignment force alignment after deleting many key_values
public native void trim(a);
// Clear the cache because of the same mechanism as SP, read the cache to load all keyvalues, memory alarm can be cleared, when used again to load all keyvalues
public native void clearMemoryCache(a);

// Get the version number
public static native String version(a);
// Get the file name that is mmapId
public native String mmapID(a);
/**** *****/ is used to lock multiple processes
/ / lock
public native void lock(a);
/ / unlock
public native void unlock(a);
// Try unlocking
public native boolean tryLock(a);

// Whether the file is valid
public static native boolean isFileValid(String var0);

/** Force a synchronous write to file **/
// In general, do not call unless you are worried about the battery
public void sync(a) {
   this.sync(true);
}
//
public void async(a) {
   this.sync(false);
}
//
private native void sync(boolean var1);
/*** Shared memory related ***/
public native int ashmemFD(a);

public native int ashmemMetaFD(a);

/** When a String or byte[] is fetched from MMKV, there will be a memory copy from native to JVM. If this value is immediately passed to another native library (JNI), there will be another memory copy from the JVM to Native. When this value is large, the whole process can be very wasteful. * The Native Buffer is designed to solve this problem. *Native Buffer is a memory Buffer created by Native, encapsulated in Java as the NativeBuffer type, which can be transparently transferred to another *Native library for access and processing. The whole process avoids the waste of copying memory to and back from the JVM
public static NativeBuffer createNativeBuffer(int size) {
    long pointer = createNB(size);
    return pointer <= 0L ? null : new NativeBuffer(pointer, size);
}

public static void destroyNativeBuffer(NativeBuffer buffer) {
    destroyNB(buffer.pointer, buffer.size);
}

public int writeValueToNativeBuffer(String key, NativeBuffer buffer) {
    return this.writeValueToNB(this.nativeHandle, key, buffer.pointer, buffer.size);
}
// Multi-process notification listener
private static native void setWantsContentChangeNotify(boolean var0);

public native void checkContentChangedByOuterProcess(a);
Copy the code

5. Write/read data

Support types: bool/float/string, byte [] / int/long/double/Set/Parcelable

// Read/write bool Type correlation
public boolean encode(String key, boolean value) {
    return this.encodeBool(this.nativeHandle, key, value);
}
public boolean decodeBool(String key) {
    return this.decodeBool(this.nativeHandle, key, false);
}

public boolean decodeBool(String key, boolean defaultValue) {
    return this.decodeBool(this.nativeHandle, key, defaultValue);
}
// Read and write int types
public boolean encode(String key, int value) {
    return this.encodeInt(this.nativeHandle, key, value);
}

public int decodeInt(String key) {
    return this.decodeInt(this.nativeHandle, key, 0);
}

public int decodeInt(String key, int defaultValue) {
    return this.decodeInt(this.nativeHandle, key, defaultValue);
}
// Read and write long
public boolean encode(String key, long value) {
    return this.encodeLong(this.nativeHandle, key, value);
}

public long decodeLong(String key) {
    return this.decodeLong(this.nativeHandle, key, 0L);
}

public long decodeLong(String key, long defaultValue) {
    return this.decodeLong(this.nativeHandle, key, defaultValue);
}
// Read and write float types
public boolean encode(String key, float value) {
    return this.encodeFloat(this.nativeHandle, key, value);
}

public float decodeFloat(String key) {
    return this.decodeFloat(this.nativeHandle, key, 0.0 F);
}

public float decodeFloat(String key, float defaultValue) {
    return this.decodeFloat(this.nativeHandle, key, defaultValue);
}
// Read and write double
public boolean encode(String key, double value) {
    return this.encodeDouble(this.nativeHandle, key, value);
}

public double decodeDouble(String key) {
    return this.decodeDouble(this.nativeHandle, key, 0.0 D);
}

public double decodeDouble(String key, double defaultValue) {
    return this.decodeDouble(this.nativeHandle, key, defaultValue);
}
// Read and write string types
public boolean encode(String key, @Nullable String value) {
    return this.encodeString(this.nativeHandle, key, value);
}

@Nullable
public String decodeString(String key) {
    return this.decodeString(this.nativeHandle, key, (String)null);
}

@Nullable
public String decodeString(String key, @Nullable String defaultValue) {
    return this.decodeString(this.nativeHandle, key, defaultValue);
}
// Read the set
      
        type
      
public boolean encode(String key, @Nullable Set<String> value) {
    return this.encodeSet(this.nativeHandle, key, value == null ? null : (String[])value.toArray(new String[0]));
}

@Nullable
public Set<String> decodeStringSet(String key) {
    return this.decodeStringSet(key, (Set)null);
}

@Nullable
public Set<String> decodeStringSet(String key, @Nullable Set<String> defaultValue) {
    return this.decodeStringSet(key, defaultValue, HashSet.class);
}

@Nullable
public Set<String> decodeStringSet(String key, @Nullable Set<String> defaultValue, Class<? extends Set> cls) {
    String[] result = this.decodeStringSet(this.nativeHandle, key);
    if (result == null) {
        return defaultValue;
    } else {
        Set a;
        try {
            a = (Set)cls.newInstance();
        } catch (IllegalAccessException var7) {
            return defaultValue;
        } catch (InstantiationException var8) {
            return defaultValue;
        }

        a.addAll(Arrays.asList(result));
        returna; }}/ / read/write byte []
public boolean encode(String key, @Nullable byte[] value) {
    return this.encodeBytes(this.nativeHandle, key, value);
}

@Nullable
public byte[] decodeBytes(String key) {
    return this.decodeBytes(key, (byte[])null);
}

@Nullable
public byte[] decodeBytes(String key, @Nullable byte[] defaultValue) {
    byte[] ret = this.decodeBytes(this.nativeHandle, key);
    returnret ! =null ? ret : defaultValue;
}
// Read and write the serialization type Parcelable
public boolean encode(String key, @Nullable Parcelable value) {
    if (value == null) {
        return this.encodeBytes(this.nativeHandle, key, (byte[])null);
    } else {
        Parcel source = Parcel.obtain();
        value.writeToParcel(source, value.describeContents());
        byte[] bytes = source.marshall();
        source.recycle();
        return this.encodeBytes(this.nativeHandle, key, bytes); }}@Nullable
public <T extends Parcelable> T decodeParcelable(String key, Class<T> tClass) {
    return this.decodeParcelable(key, tClass, (Parcelable)null);
}

@Nullable
public <T extends Parcelable> T decodeParcelable(String key, Class<T> tClass, @Nullable T defaultValue) {
    if (tClass == null) {
        return defaultValue;
    } else {
        byte[] bytes = this.decodeBytes(this.nativeHandle, key);
        if (bytes == null) {
            return defaultValue;
        } else {
            Parcel source = Parcel.obtain();
            source.unmarshall(bytes, 0, bytes.length);
            source.setDataPosition(0);

            Parcelable var8;
            try {
                String name = tClass.toString();
                Creator creator;
                synchronized(mCreators) {
                    // Get it from the local cache
                    creator = (Creator)mCreators.get(name);
                    if (creator == null) {
                        // Get the public variable CREATOR. Since it is reflection, it will be saved in the HashMap to avoid time consuming
                        Field f = tClass.getField("CREATOR");
                        creator = (Creator)f.get((Object)null);
                        if(creator ! =null) { mCreators.put(name, creator); }}}// Null indicates that the class does not implement Parcelable serialization
                if (creator == null) {
                    throw new Exception("Parcelable protocol requires a non-null static Parcelable.Creator object called CREATOR on class " + name);
                }

                var8 = (Parcelable)creator.createFromParcel(source);
            } catch (Exception var16) {
                simpleLog(MMKVLogLevel.LevelError, var16.toString());
                return defaultValue;
            } finally {
                source.recycle();
            }

            returnvar8; }}}/ / support types: bool/int, byte [] / long/float/double/Parcelable/Set < String > / String;
// The key to reading and writing data is that nativeHandle is like a file handle
Copy the code

6. Sp migration

public int importFromSharedPreferences(SharedPreferences preferences) {
    // get all key pairs stored in spMap<String, ? > kvs = preferences.getAll();if(kvs ! =null && kvs.size() > 0) {
        Iterator var3 = kvs.entrySet().iterator();

        while(var3.hasNext()) { Entry<String, ? > entry = (Entry)var3.next(); String key = (String)entry.getKey(); Object value = entry.getValue();if(key ! =null&& value ! =null) {
                if (value instanceof Boolean) {
                    this.encodeBool(this.nativeHandle, key, (Boolean)value);
                } else if (value instanceof Integer) {
                    this.encodeInt(this.nativeHandle, key, (Integer)value);
                } else if (value instanceof Long) {
                    this.encodeLong(this.nativeHandle, key, (Long)value);
                } else if (value instanceof Float) {
                    this.encodeFloat(this.nativeHandle, key, (Float)value);
                } else if (value instanceof Double) {
                    this.encodeDouble(this.nativeHandle, key, (Double)value);
                } else if (value instanceof String) {
                    this.encodeString(this.nativeHandle, key, (String)value);
                } else if (value instanceof Set) {
                    this.encode(key, (Set)value);
                } else {
                    simpleLog(MMKVLogLevel.LevelError, "unknown type: "+ value.getClass()); }}}// Return the number of SP key-value pairs
        return kvs.size();
    } else {
        return 0; }}//sp data change monitor here MMKV is not implemented, call will throw exception cautious call
 public void registerOnSharedPreferenceChangeListener(OnSharedPreferenceChangeListener listener) {
        throw new UnsupportedOperationException("Not implement in MMKV");
    }

    public void unregisterOnSharedPreferenceChangeListener(OnSharedPreferenceChangeListener listener) {
        throw new UnsupportedOperationException("Not implement in MMKV");
    }


Copy the code

7. Log direction and data recovery

// The default log level is levelInfo, converted to the corresponding level
private static int logLevel2Int(MMKVLogLevel level) {
    byte realLevel;
    switch(level) {
    case LevelDebug:
        realLevel = 0;
        break;
    case LevelWarning:
        realLevel = 2;
        break;
    case LevelError:
        realLevel = 3;
        break;
    case LevelNone:
        realLevel = 4;
        break;
    case LevelInfo:
    default:
        realLevel = 1;
    }

    return realLevel;
}

public static void setLogLevel(MMKVLogLevel level) {
    int realLevel = logLevel2Int(level);
    // set it to the MMKV base library
    setLogLevel(realLevel);
}
/ / register gCallbackHandler
public static void registerHandler(MMKVHandler handler) {
    gCallbackHandler = handler;
    if (gCallbackHandler.wantLogRedirecting()) {
        setCallbackHandler(true.true);
        gWantLogReDirecting = true;
    } else {
        setCallbackHandler(false.true);
        gWantLogReDirecting = false; }}// Unregister gCallbackHandler
public static void unregisterHandler(a) {
    gCallbackHandler = null;
    setCallbackHandler(false.false);
    gWantLogReDirecting = false;
}

If the gCallbackHandler is configured, the configured policy is used. You can either discard the file or recover the file. The recovery is not trusted
private static int onMMKVCRCCheckFail(String mmapID) {
    MMKVRecoverStrategic strategic = MMKVRecoverStrategic.OnErrorDiscard;
    if(gCallbackHandler ! =null) {
        strategic = gCallbackHandler.onMMKVCRCCheckFail(mmapID);
    }

    simpleLog(MMKVLogLevel.LevelInfo, "Recover strategic for " + mmapID + " is " + strategic);
    Integer value = (Integer)recoverIndex.get(strategic);
    return value == null ? 0 : value;
}
If the length of a file is incorrect, the discard policy is adopted by default. If gCallbackHandler is set, the discard policy is adopted. The recovery is not trusted
private static int onMMKVFileLengthError(String mmapID) {
    MMKVRecoverStrategic strategic = MMKVRecoverStrategic.OnErrorDiscard;
    if(gCallbackHandler ! =null) {
        strategic = gCallbackHandler.onMMKVFileLengthError(mmapID);
    }

    simpleLog(MMKVLogLevel.LevelInfo, "Recover strategic for " + mmapID + " is " + strategic);
    Integer value = (Integer)recoverIndex.get(strategic);
    return value == null ? 0 : value;
}
// If gCallbackHandler sets Log redirection, the service side takes over the redirection Log. Otherwise, MMKV uses the default Log output
private static void mmkvLogImp(int level, String file, int line, String function, String message) {
    if(gCallbackHandler ! =null && gWantLogReDirecting) {
        gCallbackHandler.mmkvLog(index2LogLevel[level], file, line, function, message);
    } else {
        switch(index2LogLevel[level]) {
        case LevelDebug:
            Log.d("MMKV", message);
            break;
        case LevelWarning:
            Log.w("MMKV", message);
            break;
        case LevelError:
            Log.e("MMKV", message);
        case LevelNone:
        default:
            break;
        case LevelInfo:
            Log.i("MMKV", message); }}}// Get the stack information organization log function does not do the out-of-bounds judgment
private static void simpleLog(MMKVLogLevel level, String message) {
    StackTraceElement[] stacktrace = Thread.currentThread().getStackTrace();
    StackTraceElement e = stacktrace[stacktrace.length - 1];
    Integer i = (Integer)logLevel2Index.get(level);
    int intLevel = i == null ? 0 : i;
    mmkvLogImp(intLevel, e.getFileName(), e.getLineNumber(), e.getMethodName(), message);
}
Copy the code
8. Data change notification (interprocess use)
// Register data change listener
public static void registerContentChangeNotify(MMKVContentChangeNotification notify) { gContentChangeNotify = notify; setWantsContentChangeNotify(gContentChangeNotify ! =null);
}
// Unregister data change listener
public static void unregisterContentChangeNotify(a) {
    gContentChangeNotify = null;
    setWantsContentChangeNotify(false);
}
// Notify other processes when data changes
private static void onContentChangedByOuterProcess(String mmapID) {
    if(gContentChangeNotify ! =null) { gContentChangeNotify.onContentChangedByOuterProcess(mmapID); }}private static native void setWantsContentChangeNotify(boolean var0);
Copy the code

Precautions for use of MMKV

Through reading the source code or consulting materials, sort out the problems and debugging tools in the process of using MMKV, as preparation before access

Q1. Which access method can reduce the volume of APK

Implementation ‘com. Tencent: MMKV :1.2.7’
Implementation ‘com. Tencent :mmkv-static:1.2.7’

Libc++ _shared.so/libc++. So/libc++ so/libc++. You can see that the downloadSize is also reduced by 200K;

However, MMKV official notes recommend non-static access to reduce the size of the installation package. To avoid conflicts with underlying libraries, static access is recommended

Q2. Is MMKV thread safe?

MMKV does not guarantee atomicity, so it is not thread-safe. The MMKV principle is incremental updating, meaning that ideally the last key written is the final result. Therefore, use caution when updating the same key for the same file concurrently

Q3. Is MMKV type safe?

Use MMKV to store a string of type “123”, and then use other types of data, can get a different value, but do not report an error, do not take the default value (the same with other types).

Q4, checkProcessMode?

Source code analysis see source code analysis, here will not repeat, put forward the following questions:

1. For the first time, mapId is used to obtain MMKV in a single process, and Handle has been stored in set; Is the handle obtained by using the same mapId and multi-process for the second time the same as before?
Case1: first use a single process with ID = “singleMMkvProcess” to obtain MMKV without killing APP, then use multiple processes with ID = “singleMMkvProcess” to obtain MMKV, and check whether the acquired MMKV instance Handles are consistent
Result: The handle is consistent
Case2: firstly, multiple processes with ID = “singleMMkvProcess” are used to obtain MMKV without killing APP. Then, a single process with ID = “singleMMkvProcess” is used to obtain MMKV, and the obtained MMKV instance Handles are consistent
Result: The handle is consistent
Case3: first use a single process with ID = “singleMMkvProcess” to obtain MMKV and kill app, then use multiple processes with ID = “singleMMkvProcess” to obtain MMKV and check whether the obtained MMKV instance Handles are consistent
Results: Inconsistent
Case4: First use multiple processes with ID = “singleMMkvProcess” to obtain MMKV and kill APP. Then use a single process with ID = “singleMMkvProcess” to obtain MMKV and check whether the obtained MMKV instance Handles are consistent
Result: The handle is consistent
2. If I use mapId for the first time, get MMKV with the secret key; If the same mapId is used for the second time without the secret key, is the handle obtained the same as before?
Case1: first use id= “singleMMkvProcess” to obtain MMKV without adding the secret key, without killing app. Then use id= “singleMMkvProcess” to add the secret key to obtain MMKV, and check whether the obtained MMKV instance Handles are consistent
Result: The handle is consistent
Case2: first use id= “singleMMkvProcess” to add the secret key to obtain MMKV without killing APP, then use ID = “singleMMkvProcess” to obtain MMKV without adding the secret key, and check whether the obtained MMKV instance Handles are consistent
Result: The handle is consistent
Case3: first use id= “singleMMkvProcess” to obtain MMKV without secret key, kill APP, then use id= “singleMMkvProcess” to add secret key to obtain MMKV, and check whether the obtained MMKV instance Handles are consistent
Result: The handle is consistent
Case4: first use ID = “singleMMkvProcess” to encrypt and obtain MMKV and kill app. Then use ID = “singleMMkvProcess” to obtain MMKV without encryption. Are the obtained MMKV instance Handles consistent
Result: Handle is inconsistent
MMKV is encrypted and stored first, and rekey is not encrypted. In the case that APP is not killed, the obtained results are consistent because handle is obtained consistently. In the case of killing APP, the fourth handle is inconsistent and the written value cannot be obtained
3. Note: Try to avoid inserting the secret key or changing the process mode to obtain MMKV during the process

Q5, multi-process anonymous shared memory pit?

Anonymous shared memory (use with caution)
public static MMKV mmkvWithAshmemID(Context context, String mmapID, int size, int mode, @Nullable String cryptKey)
Copy the code

How it works: The internal implementation actually uses MMKVContentProvider to pass file descriptors

Example: In mmkvdemo, use mmkvWithAshmemID to create an MMKV instance. The main process also creates an MMKV instance in two service processes. In this case, the main process writes a key value, and the two service processes read the key value correctly.

Android Stuido console kill the MMkvdemo process (two service processes are still in use), then open the app to do the same process, I found that in fact the main process MMKV instances and two service processes MMKV instances map the same memory address, the main process changed the key value, The server process read is still old.

For multi-process applications, it is common for the main process to be killed in the background, so this scenario has usage drawbacks

Q6. Is it convenient to migrate to other storage?

Due to type erasure during storage, all data cannot be obtained at one time and subsequent one-key migration like SP cannot be achieved

publicMap<String, ? > getAll() {throw new UnsupportedOperationException("use allKeys() instead, getAll() not implement because type-erasure inside mmkv");
}
Copy the code

Q7. Is the data store divided into two files?

1. The two files increase the probability of file verification failure

2. The data volume is small and occupies 4Kb space