Public number: byte array, hope to help you 馃ぃ馃ぃ
SharedPreferences is a persistent storage solution provided by the system for storing a small number of key-value pairs of data. It is simple in structure and easy to use, and almost all applications can use it. On the other hand, SharedPreferences also has a lot of problems, among which ANR problems are common. Bytedance technical team once published an article specifically to explain this problem: analyzing ANR problems caused by SharedPreference Apply. Now, Google Jetpack has also introduced a new persistent storage solution, DataStore, that is likely to replace SharedPreferences
This article is combined with the source code to analyze the defects of SharedPreferences and the specific reasons behind it, based on SDK 30 analysis, so that readers do know what it is and what it is, and finally introduced my personal storage mechanism design scheme, I hope to help you 馃ぃ馃ぃ
Have to say the pit
SP data will always occupy memory
SharedPreferences itself is an interface whose implementation class is SharedPreferencesImpl, and the methods of the Context that are related to SharedPreferences are implemented by ContextImpl. Each SP in our project more or less holds key-value data, and whenever we acquire a SharedPreferences object, its key-value data is kept in memory until the application process is terminated. Because each SharedPreferences object is cached by the system as a static variable, corresponding to the static variable sSharedPrefsCache in ContextImpl
class ContextImpl extends Context {
Cache all SharedPreferences based on the application package name, based on xmlFile and the specific SharedPreferencesImpl
private static ArrayMap<String, ArrayMap<File, SharedPreferencesImpl>> sSharedPrefsCache;
// Retrieve the corresponding xmlFile according to fileName
private ArrayMap<String, File> mSharedPrefsPaths;
}
Copy the codeEach SP corresponds to an xmlFile on the local disk, fileName is explicitly specified by the developer, and each xmlFile corresponds to a SharedPreferencesImpl. So the logic for ContextImpl is to get the xmlFile from fileName and then the SharedPreferencesImpl from xmlFile, Eventually, all SharedPreferencesImpl within the application will be cached in the static variable sSharedPrefsCache
In addition, since SharedPreferencesImpl automatically loads all key-value pairs in xmlFile upon initialization, and ContextImpl does not see any logic inside to clean the sSharedPrefsCache cache, As a result, sSharedPrefsCache will remain in memory until the end of the process, and its memory size will increase as more SharedPreferences are referenced, which may continue to occupy a significant portion of memory
@Override
public SharedPreferences getSharedPreferences(String name, int mode) {... the File File;synchronized (ContextImpl.class) {
if (mSharedPrefsPaths == null) {
mSharedPrefsPaths = new ArrayMap<>();
}
file = mSharedPrefsPaths.get(name);
if (file == null) { file = getSharedPreferencesPath(name); mSharedPrefsPaths.put(name, file); }}return getSharedPreferences(file, mode);
}
@Override
public SharedPreferences getSharedPreferences(File file, int mode) {
SharedPreferencesImpl sp;
synchronized (ContextImpl.class) {
final ArrayMap<File, SharedPreferencesImpl> cache = getSharedPreferencesCacheLocked();
sp = cache.get(file);
if (sp == null) {路路路 sp =new SharedPreferencesImpl(file, mode);
cache.put(file, sp);
returnsp; }}...return sp;
}
@GuardedBy("ContextImpl.class")
private ArrayMap<File, SharedPreferencesImpl> getSharedPreferencesCacheLocked(a) {
if (sSharedPrefsCache == null) {
sSharedPrefsCache = new ArrayMap<>();
}
final String packageName = getPackageName();
ArrayMap<File, SharedPreferencesImpl> packagePrefs = sSharedPrefsCache.get(packageName);
if (packagePrefs == null) {
packagePrefs = new ArrayMap<>();
sSharedPrefsCache.put(packageName, packagePrefs);
}
return packagePrefs;
}
Copy the codeGetValue can cause a thread to block
The SharedPreferencesImpl constructor starts a child thread directly to load the disk file, which means that the operation is asynchronous (I’m talking) and can take a while to complete if the file is large or the thread scheduling system doesn’t start the thread right away
final class SharedPreferencesImpl implements SharedPreferences {
@UnsupportedAppUsage
SharedPreferencesImpl(File file, int mode) {
mFile = file;
mBackupFile = makeBackupFile(file);
mMode = mode;
mLoaded = false;
mMap = null;
mThrowable = null;
startLoadFromDisk();
}
@UnsupportedAppUsage
private void startLoadFromDisk(a) {
synchronized (mLock) {
mLoaded = false;
}
new Thread("SharedPreferencesImpl-load") {
public void run(a) {
// Load the disk fileloadFromDisk(); } }.start(); }}Copy the codeIf we initialize SharedPreferencesImpl and then go to getValue, we’ll need to make sure that the child thread is loaded. The SharedPreferencesImpl determines whether an external thread needs to be blocked by calling awaitLoadedLocked() in each getValue method, ensuring that the value is not executed until the child thread has finished executing. The loadFromDisk() method calls mLock.notifyAll() to wake up all blocked threads after the task is complete
So, if SharedPreferences store a large amount of data, it can cause external caller threads to block, and in severe cases, ANR. Of course, this is only possible until the loading of the disk file is complete, when awaitLoadedLocked() does not naturally block the thread
@Override
@Nullable
public String getString(String key, @Nullable String defValue) {
synchronized (mLock) {
// Determine if an external thread needs to wait
awaitLoadedLocked();
String v = (String)mMap.get(key);
returnv ! =null? v : defValue; }}@GuardedBy("mLock")
private void awaitLoadedLocked(a) {
if(! mLoaded) {// Raise an explicit StrictMode onReadFromDisk for this
// thread, since the real read will be in a different
// thread and otherwise ignored by StrictMode.
BlockGuard.getThreadPolicy().onReadFromDisk();
}
while(! mLoaded) {try {
// The thread has not been loaded yet
mLock.wait();
} catch (InterruptedException unused) {
}
}
if(mThrowable ! =null) {
throw newIllegalStateException(mThrowable); }}private void loadFromDisk(a) {...synchronized (mLock) {
mLoaded = true;
mThrowable = thrown;
// It's important that we always signal waiters, even if we'll make
// them fail with an exception. The try-finally is pretty wide, but
// better safe than sorry.
try {
if (thrown == null) {
if(map ! =null) {
mMap = map;
mStatTimestamp = stat.st_mtim;
mStatSize = stat.st_size;
} else {
mMap = newHashMap<>(); }}// In case of a thrown exception, we retain the old map. That allows
// any open editors to commit and store updates.
} catch (Throwable t) {
mThrowable = t;
} finally {
// Wake up all blocked threadsmLock.notifyAll(); }}}Copy the codeGetValue does not guarantee data type safety
The following code during compilation phase is completely normal, but the runtime will throw an exception: Java. Lang. ClassCastException: Java. Lang. Integer always be cast to Java. Lang. String. SharedPreferences has no way of limiting this operation, and requires the developer’s own code specification to do so
val sharedPreferences: SharedPreferences = getSharedPreferences("UserInfo", Context.MODE_PRIVATE)
val key = "userName"
val edit = sharedPreferences.edit()
edit.putInt(key, 11)
edit.apply()
val name = sharedPreferences.getString(key, "")
Copy the codeSP does not support multi-process data sharing
When creating SharedPreferences, you need to pass in a mode flag bit parameter of int type. There is a MODE_MULTI_PROCESS flag bit that is related to multiple processes. This flag bit can guarantee multi-process data synchronization to a certain extent, but does not play a significant role. It does not guarantee the security of multi-process concurrency
val sharedPreferences: SharedPreferences = getSharedPreferences("UserInfo", Context.MODE_MULTI_PROCESS)
Copy the codeAs mentioned above, the SharedPreferencesImpl will remain in memory after loading, and each fetch will use the cache data directly, usually without loading the disk file again. MODE_MULTI_PROCESS determines whether the current disk file has been modified relative to the last memory modification, and reloads the disk file whenever it attempts to obtain the SharedPreferences instance. Thus, certain data synchronization can be achieved in multi-process environment
However, this synchronization itself has little effect, because even if the disk files are reloaded, the memory data in different processes will not be synchronized in real time when the SP value is changed. In addition, when multiple processes change the SP value at the same time, data loss and data overwriting may occur. Therefore, SharedPreferences does not support multi-process data sharing, MODE_MULTI_PROCESS is deprecated, and its annotations recommend using ContentProvider for cross-process communication
class ContextImpl extends Context {
@Override
public SharedPreferences getSharedPreferences(File file, int mode) {
SharedPreferencesImpl sp;
synchronized(ContextImpl. Class) {...}if((mode & Context.MODE_MULTI_PROCESS) ! =0 ||
getApplicationInfo().targetSdkVersion < android.os.Build.VERSION_CODES.HONEYCOMB) {
// If somebody else (some other process) changed the prefs
// file behind our back, we reload it. This has been the
// historical (if undocumented) behavior.
// Reload the disk file
sp.startReloadIfChangedUnexpectedly();
}
returnsp; }}Copy the codeSP does not support incremental updates
As we know, there are two methods for submitting data in SharedPreferences: Commit () and apply(), which correspond to synchronous and asynchronous changes respectively, correspond to full updates. SharedPreferences change the file in the smallest unit, even if we change only one key-value pair. Both methods also re-write all key-value pair data to disk files, meaning that SharedPreferences only supports full updates
The Editor object is the SharedPreferencesImpl inner class EditorImpl. Each putValue method of the EditorImpl saves the key-value passed in in mModified. No file changes have been made yet. The special methods are remove and clear. The remove method will take this as the value of the key-value pair, and then it will determine whether to remove or modify the key-value pair by comparing the equality of values. The clear method simply marks mClear to true
public final class EditorImpl implements Editor {
private final Object mEditorLock = new Object();
@GuardedBy("mEditorLock")
private final Map<String, Object> mModified = new HashMap<>();
@GuardedBy("mEditorLock")
private boolean mClear = false;
@Override
public Editor putString(String key, @Nullable String value) {
synchronized (mEditorLock) {
mModified.put(key, value);
return this; }}@Override
public Editor remove(String key) {
synchronized (mEditorLock) {
// Store the current EditorImpl object
mModified.put(key, this);
return this; }}@Override
public Editor clear(a) {
synchronized (mEditorLock) {
mClear = true;
return this; }}}Copy the codeBoth commit() and apply() methods get the full modified data by calling commitToMemory()
CommitToMemory () uses the DIff algorithm, all key-pair data contained in SharedPreferences is stored in mapToWriteToDisk, and all key-pair data changed to Editor is stored in mModified. If mClear is true, mapToWriteToDisk is cleared and mModified is iterated to synchronize all changes in mModified to mapToWriteToDisk. Eventually mapToWriteToDisk stores the full amount of data to be rewritten to the disk file, and SharedPreferences completely overwrite the old XML file based on mapToWriteToDisk
// Returns true if any changes were made
private MemoryCommitResult commitToMemory(a) {
long memoryStateGeneration;
boolean keysCleared = false;
List<String> keysModified = null;
Set<OnSharedPreferenceChangeListener> listeners = null;
Map<String, Object> mapToWriteToDisk;
synchronized (SharedPreferencesImpl.this.mLock) {
// We optimistically don't make a deep copy until
// a memory commit comes in when we're already
// writing to disk.
if (mDiskWritesInFlight > 0) {
// We can't modify our mMap as a currently
// in-flight write owns it. Clone it before
// modifying it.
// noinspection unchecked
mMap = new HashMap<String, Object>(mMap);
}
// Get the full data in memory
mapToWriteToDisk = mMap;
mDiskWritesInFlight++;
boolean hasListeners = mListeners.size() > 0;
if (hasListeners) {
keysModified = new ArrayList<String>();
listeners = new HashSet<OnSharedPreferenceChangeListener>(mListeners.keySet());
}
synchronized (mEditorLock) {
// used to mark whether mapToWriteToDisk has finally changed
boolean changesMade = false;
if (mClear) {
if(! mapToWriteToDisk.isEmpty()) { changesMade =true;
// Clear all data in memory
mapToWriteToDisk.clear();
}
keysCleared = true;
// Restore the status to avoid status misalignment during secondary modification
mClear = false;
}
for (Map.Entry<String, Object> e : mModified.entrySet()) {
String k = e.getKey();
Object v = e.getValue();
// "this" is the magic value for a removal mutation. In addition,
// setting a value to "null" for a given key is specified to be
// equivalent to calling remove on that key.
if (v == this || v == null) { // means to remove the key-value pair
if(! mapToWriteToDisk.containsKey(k)) {continue;
}
mapToWriteToDisk.remove(k);
} else { // Change the key-value pairs
if (mapToWriteToDisk.containsKey(k)) {
Object existingValue = mapToWriteToDisk.get(k);
if(existingValue ! =null && existingValue.equals(v)) {
continue; }}// The value needs to be saved only if the key-value pair is actually modified or newly inserted
mapToWriteToDisk.put(k, v);
}
changesMade = true;
if(hasListeners) { keysModified.add(k); }}// Restore the status to avoid status misalignment during secondary modification
mModified.clear();
if(changesMade) { mCurrentMemoryStateGeneration++; } memoryStateGeneration = mCurrentMemoryStateGeneration; }}return new MemoryCommitResult(memoryStateGeneration, keysCleared, keysModified,
listeners, mapToWriteToDisk);
}
Copy the codeThe counterintuitive use of clear
Look at the following example. Semantically, it would be intuitive to end up with only one key-value pair left in the SharedPreferences, but in fact both key-value pairs will be retained, and only the two key-value pairs will be retained
val sharedPreferences: SharedPreferences = getSharedPreferences("UserInfo", Context.MODE_PRIVATE)
val edit = sharedPreferences.edit()
edit.putString("name"."Yip Chi Chan").clear().putString("blog"."https://juejin.cn/user/923245496518439")
edit.apply()
Copy the codeThe cause of this problem also needs to be seen with the commitToMemory() method. Clear () sets mClear to true, so mapToWriteToDisk is cleared of all key-value pairs in memory during the first step. When the second step is performed, all data in mModified is synchronized to mapToWriteToDisk, resulting in both the name and blog key pairs being preserved and all other key pairs removed
Therefore, editor.clear () should not be called coherently before putValue statements, which can cause a discrepancy between understanding and actual effect
// Returns true if any changes were made
private MemoryCommitResult commitToMemory(a) {
long memoryStateGeneration;
boolean keysCleared = false;
List<String> keysModified = null;
Set<OnSharedPreferenceChangeListener> listeners = null;
Map<String, Object> mapToWriteToDisk;
synchronized (SharedPreferencesImpl.this.mLock) {
// We optimistically don't make a deep copy until
// a memory commit comes in when we're already
// writing to disk.
if (mDiskWritesInFlight > 0) {
// We can't modify our mMap as a currently
// in-flight write owns it. Clone it before
// modifying it.
// noinspection unchecked
mMap = new HashMap<String, Object>(mMap);
}
// Get the full data in memory
mapToWriteToDisk = mMap;
mDiskWritesInFlight++;
boolean hasListeners = mListeners.size() > 0;
if (hasListeners) {
keysModified = new ArrayList<String>();
listeners = new HashSet<OnSharedPreferenceChangeListener>(mListeners.keySet());
}
synchronized (mEditorLock) {
boolean changesMade = false;
if (mClear) { / / the first step
if(! mapToWriteToDisk.isEmpty()) { changesMade =true;
// Clear all data in memory
mapToWriteToDisk.clear();
}
keysCleared = true;
// Restore the status to avoid status misalignment during secondary modification
mClear = false;
}
for (Map.Entry<String, Object> e : mModified.entrySet()) { / / the second step
String k = e.getKey();
Object v = e.getValue();
// "this" is the magic value for a removal mutation. In addition,
// setting a value to "null" for a given key is specified to be
// equivalent to calling remove on that key.
if (v == this || v == null) { // means to remove the key-value pair
if(! mapToWriteToDisk.containsKey(k)) {continue;
}
mapToWriteToDisk.remove(k);
} else { // Change the key-value pairs
if (mapToWriteToDisk.containsKey(k)) {
Object existingValue = mapToWriteToDisk.get(k);
if(existingValue ! =null && existingValue.equals(v)) {
continue; }}// The value needs to be saved only if the key-value pair is actually modified or newly inserted
mapToWriteToDisk.put(k, v);
}
changesMade = true;
if(hasListeners) { keysModified.add(k); }}// Restore the status to avoid status misalignment during secondary modification
mModified.clear();
if(changesMade) { mCurrentMemoryStateGeneration++; } memoryStateGeneration = mCurrentMemoryStateGeneration; }}return new MemoryCommitResult(memoryStateGeneration, keysCleared, keysModified,
listeners, mapToWriteToDisk);
}
Copy the codeCommit and Applay may cause ANR
Commit () takes full data via commitToMemory(), a MemoryCommitResult, and then commits to enqueueDiskWrite writing full data to a disk file. The caller thread is blocked by CountDownLatch until the write completes, and the method returns the successful status of the change
@Override
public boolean commit(a) {
long startTime = 0;
if (DEBUG) {
startTime = System.currentTimeMillis();
}
// Get the full data after modification
MemoryCommitResult mcr = commitToMemory();
// Submit the task to write the disk file
SharedPreferencesImpl.this.enqueueDiskWrite(
mcr, null /* sync write on this thread okay */);
try {
// Block and wait until the XML file is written (successfully or not)
mcr.writtenToDiskLatch.await();
} catch (InterruptedException e) {
return false;
} finally {
if (DEBUG) {
Log.d(TAG, mFile.getName() + ":" + mcr.memoryStateGeneration
+ " committed after " + (System.currentTimeMillis() - startTime)
+ " ms");
}
}
notifyListeners(mcr);
return mcr.writeToDiskResult;
}
Copy the codeThe enqueueDiskWrite method is the one that contains the actual disk write logic. Since it is possible for multiple threads to execute apply() and commit() simultaneously, the corresponding disk file is only one. Therefore, the enqueueDiskWrite method must ensure that the write operation is orderly, avoiding data loss or overwriting, or even file corruption
The specific logic of the enqueueDiskWrite method:
- WriteToDiskRunnable uses the internal lock mWritingToDiskLock to ensure that writeToFile operations are organized and avoid multiple threads
- For the COMMIT operation, if only one thread is currently committing the changes, writeToDiskRunnable is executed directly on that thread and the process ends
- In other cases (apply, multi-threaded simultaneous commit, or apply) writeToDiskRunnable is committed to QueuedWork
- QueuedWork uses handlerThreads internally to perform writeToDiskRunnable, and handlerThreads themselves ensure that multiple tasks are organized
private void enqueueDiskWrite(final MemoryCommitResult mcr,
final Runnable postWriteRunnable) {
final boolean isFromSyncCommit = (postWriteRunnable == null);
final Runnable writeToDiskRunnable = new Runnable() {
@Override
public void run(a) {
synchronized (mWritingToDiskLock) {
// Write the disk file
writeToFile(mcr, isFromSyncCommit);
}
synchronized (mLock) {
mDiskWritesInFlight--;
}
if(postWriteRunnable ! =null) { postWriteRunnable.run(); }}};// Typical #commit() path with fewer allocations, doing a write on
// the current thread.
if (isFromSyncCommit) { // The commit() method will walk in
boolean wasEmpty = false;
synchronized (mLock) {
wasEmpty = mDiskWritesInFlight == 1;
}
if (wasEmpty) {
If wasEmpty is true, only one thread is currently performing the commit, so the task is completed directly on that thread
writeToDiskRunnable.run();
return; } } QueuedWork.queue(writeToDiskRunnable, ! isFromSyncCommit); }Copy the codeIn addition, there is a more important knowledge points need to pay attention to. The writeToFile method verifies this task to avoid multiple invalid disk tasks. , mDiskStateGeneration is representative of the last successful written to disk file version number of tasks, mCurrentMemoryStateGeneration is the latest changes in current memory to record the version number, MCR. MemoryStateGeneration is the version number of this to perform a task. By comparing two version numbers, you can avoid repeated I/O operations caused by multiple commit or apply operations. Instead, you can only perform the last I/O operation, avoiding invalid I/O tasks
@GuardedBy("mWritingToDiskLock")
private void writeToFile(MemoryCommitResult mcr, boolean isFromSyncCommit) {...if (fileExists) {
boolean needsWrite = false;
// Only need to write if the disk state is older than this commit
// Determine the version number
if (mDiskStateGeneration < mcr.memoryStateGeneration) {
if (isFromSyncCommit) {
needsWrite = true;
} else {
synchronized (mLock) {
// No need to persist intermediate states. Just wait for the latest state to
// be persisted.
// Determine the version number
if (mCurrentMemoryStateGeneration == mcr.memoryStateGeneration) {
needsWrite = true; }}}}if(! needsWrite) {// The current version is not the latest
mcr.setDiskWriteResult(false.true);
return; }...}Copy the codeBack to the commit() method. The await() method will cause the thread to block and wait until writeToDiskRunnable is finished, regardless of whether the writeToDiskRunnable associated with the method is finally executed in the own thread or HandlerThread. This implements the effect of a commit() synchronous commit
To sum up, since SharedPreferences itself only supports full updates, even a commit() operation with a small amount of data can result in ANR if the SharedPreferences file is large
@Override
public boolean commit(a) {
long startTime = 0;
if (DEBUG) {
startTime = System.currentTimeMillis();
}
// Get the full data after modification
MemoryCommitResult mcr = commitToMemory();
// Submit the task to write the disk file
SharedPreferencesImpl.this.enqueueDiskWrite(
mcr, null /* sync write on this thread okay */);
try {
// Block and wait until the XML file is written (successfully or not)
mcr.writtenToDiskLatch.await();
} catch (InterruptedException e) {
return false;
} finally {
if (DEBUG) {
Log.d(TAG, mFile.getName() + ":" + mcr.memoryStateGeneration
+ " committed after " + (System.currentTimeMillis() - startTime)
+ " ms");
}
}
notifyListeners(mcr);
return mcr.writeToDiskResult;
}
Copy the codeFor the apply() method, the I/O operation should be committed to the child thread. It is reasonable to call enqueueDiskWrite and commit the task without waiting for the task to complete. In fact, the Apply () method is much more complex than the commit() method
The apply() method contains an awaitCommit task that blocks its execution thread until the disk task completes, and the awaitCommit is wrapped in postWriteRunnable and submitted to the enqueueDiskWrite method. The enqueueDiskWrite method will execute enqueueDiskWrite after writeToDiskRunnable
@Override
public void apply(a) {
final long startTime = System.currentTimeMillis();
final MemoryCommitResult mcr = commitToMemory();
final Runnable awaitCommit = new Runnable() {
@Override
public void run(a) {
try {
// Block the thread until the disk task completes
mcr.writtenToDiskLatch.await();
} catch (InterruptedException ignored) {
}
if (DEBUG && mcr.wasWritten) {
Log.d(TAG, mFile.getName() + ":" + mcr.memoryStateGeneration
+ " applied after " + (System.currentTimeMillis() - startTime)
+ " ms"); }}}; QueuedWork.addFinisher(awaitCommit); Runnable postWriteRunnable =new Runnable() {
@Override
public void run(a) { awaitCommit.run(); QueuedWork.removeFinisher(awaitCommit); }};// Submit the task
SharedPreferencesImpl.this.enqueueDiskWrite(mcr, postWriteRunnable);
// Okay to notify the listeners before it's hit disk
// because the listeners should always get the same
// SharedPreferences instance back, which has the
// changes reflected in memory.
notifyListeners(mcr);
}
Copy the codeWriteToDiskRunnable (writeToDiskRunnable) is executed by HandlerThread (writeToDiskRunnable), and awaitCommit (writeToDiskRunnable) is called by HandlerThread (writeToDiskRunnable). The wait for awaitCommit is strange, because awaitCommit must be called after the disk task has finished, just as HandlerThread is waiting for its own completion. In addition, HandlerThread belongs to a child thread, which should not cause the main thread ANR if it performs a time-consuming operation
To understand this, you need to look again at the ActivityThread class. When the Service and Activity life cycles are handleStopService(), handlePauseActivity(), and handleStopActivity(), The ActivityThread calls the queuedWork.waittoFinish () method
private void handleStopService(IBinder token) {
Service s = mServices.remove(token);
if(s ! =null) {
try{.../ / the keyQueuedWork.waitToFinish(); ...}catch(Exception e) {路路路}}else {
Slog.i(TAG, "handleStopService: token=" + token + " not found.");
}
//Slog.i(TAG, "Running services: " + mServices);
}
Copy the codeThe queuedWork.waittoFinish () method actively performs all disk writes and all postWriteRunnable, This results in an Activity or Service being blocked by a large number of disk writes during a life-cycle switch, resulting in ANR
public static void waitToFinish(a) {
long startTime = System.currentTimeMillis();
boolean hadMessages = false;
Handler handler = getHandler();
synchronized (sLock) {
if (handler.hasMessages(QueuedWorkHandler.MSG_RUN)) {
// Delayed work will be processed at processPendingWork() below
handler.removeMessages(QueuedWorkHandler.MSG_RUN);
if (DEBUG) {
hadMessages = true;
Log.d(LOG_TAG, "waiting"); }}// We should not delay any work as this might delay the finishers
sCanDelay = false;
}
StrictMode.ThreadPolicy oldPolicy = StrictMode.allowThreadDiskWrites();
try {
// Perform all disk write tasks
processPendingWork();
} finally {
StrictMode.setThreadPolicy(oldPolicy);
}
try {
// Execute all postWriteRunnable
while (true) {
Runnable finisher;
synchronized (sLock) {
finisher = sFinishers.poll();
}
if (finisher == null) {
break; } finisher.run(); }}finally {
sCanDelay = true;
}
synchronized (sLock) {
long waitTime = System.currentTimeMillis() - startTime;
if (waitTime > 0 || hadMessages) {
mWaitTimes.add(Long.valueOf(waitTime).intValue());
mNumWaits++;
if (DEBUG || mNumWaits % 1024= =0 || waitTime > MAX_WAIT_TIME_MILLIS) {
mWaitTimes.log(LOG_TAG, "waited: "); }}}}Copy the codeI don’t know why ActivityThread would actively trigger all disk writes, but the ByteDance team has a guess: The only reason we can guess why Google blocks the main thread to process SP before Activity and Service calls onStop is to make the data as persistent as possible. If crash occurs during the running process, SP will not be persisted. Persistence itself is an IO operation and will fail
SP plus and minus
SharedPreferencesImpl varies greatly between different system versions. For example, writeToFile is used to verify the task version number from 8.0. Prior to 8.0, I/O operations were triggered each time for consecutive COMMIT and apply, so ANR problems were more likely to recur. We need to look at each of the defects listed above in terms of the system version
It is important to note that SharedPreferences itself is positioned as a lightweight data store, designed to store simple data structures (basic data types) and provide modular partitioning for storage. Many of the “defects” described above could have been avoided if developers had strictly followed this one specification. And SharedPreferences now look a lot of problems, is also because most of the application business is much more complex than before, sometimes for convenience directly use SharedPreferences to store very complex data structure, or do not do a good job of data partitioning storage. The resulting single file is too large, which is the main cause of the problem
How to do persistence well
The following example code is probably a nightmare for many developers
val sharedPreference = getSharedPreferences("user_preference", Context.MODE_PRIVATE)
val name = sharedPreference.getString("name"."")
Copy the codeWhat’s wrong with this code? I think there are at least five:
- The strong reference to SharedPreferences results in the need for global search and replacement when the repository needs to be switched, and the workload is very heavy
- The key value is difficult to maintain and must be explicitly declared each time a value is obtained
- Poor readability, the meaning of key-value pairs can only be expressed by the key value
- Only basic data types are supported, and there is a lot of duplication in accessing custom data types. To store custom JavaBean objects to SP, the Bean object can only be converted into A Json string and stored in SP, and then manually deserialized at value time
- The data type is not clear, and you have to rely on comments to guide the developer to use the correct data type
Developers often declare various SpUtils classes for an extra layer of encapsulation, but that doesn’t solve the problem completely. There are some design flaws in SharedPreferences, but for most application developers there is not much choice but to use or not use SharedPreferences. There is not much room to solve or avoid problems with SharedPreferences, and we often have to switch to other persistent storage solutions when we encounter problems
There are two well-known persistent storage schemes: Jetpack DataStore and Tencent’S MMKV, we can of course choose to switch SharedPreferences to these two libraries in the project, but this also raises a question, if these two libraries also encounter problems or even directly abandoned in the future, Do we need to do global substitution again? How do we design to minimize the cost of each replacement? In my opinion, before introducing a new dependency library into a project, developers should be prepared to remove the library later, isolating the interface and masking the specific usage logic (of course, not every dependency library can do this). My project also used SharedPreferences to store configuration information, and then I switched to MMKV. Here is how I designed the storage structure to avoid hard coding
Current results
I classified all key-value pair data that needed to be stored in the application into three categories: user strongly associated data, application configuration data, and data that could not be changed twice. Each type of data has different storage areas and does not affect each other. The advantage of grouping data is that specific data can be removed as needed. For example, when a user logs out, only UserKVHolder should be removed, while PreferenceKVHolder and FinalKVHolder can be retained
The IKVHolder interface defines the basic access method, and MMKVKVHolder implements the specific storage logic through MMKV
// Data that is strongly bound to the user needs to be cleared when logging out, such as UserBean
// Set up the encryptKey to encrypt the storage
private val UserKVHolder: IKVHolder = MMKVKVHolder("user"."Encryption key")
// Data that is not strongly associated with the user does not need to be cleared when logging out, such as night mode and font size
private val PreferenceKVHolder: IKVHolder = MMKVKVHolder("preference")
// It is used to store data that will not change twice and is only used for historical traceability, such as the time when the application was first installed, version number, version name, etc
private val FinalKVHolder: IKVHolder = MMKVKVFinalHolder("final")
Copy the codeWe can then take advantage of Kotlin’s powerful syntactic features to define key-value pairs
For example, for data strongly associated with users, each key-value pair is defined as an attribute field of UserKV. The meaning and function of the key-value pair are identified by the attribute name, and the key of the key-value pair must be the same as the attribute name to avoid duplicate key values. Each getValue operation also supports setting a default value. IKVHolder internally through Gson to achieve serialization and deserialization, so UserKV can directly store Javabeans, JavaBeanList, Map and other data structures
object UserKV : IKVHolder by UserKVHolder {
var name: String
get() = get("name"."")
set(value) = set("name", value)
var blog: String
get() = get("blog"."")
set(value) = set("blog", value)
var userBean: UserBean?
get() = getBeanOrNull("userBean")
set(value) = set("userBean", value)
var userBeanOfDefault: UserBean
get() = getBeanOrDefault(
"userBeanOfDefault",
UserBean("Yip Chi Chan"."https://juejin.cn/user/923245496518439"))set(value) = set("userBeanOfDefault", value)
var userBeanList: List<UserBean>
get() = getBean("userBeanList")
set(value) = set("userBeanList", value)
var map: Map<Int, String>
get() = getBean("map")
set(value) = set("map", value)
}
Copy the codeIn addition, we can check the value in the setValue method to avoid invalid values
object UserKV : IKVHolder by UserKVHolder {
var age: Int
get() = get("age".0)
set(value) {
if (value <= 0) {
return
}
set("age", value)
}
}
Copy the codeAfter that, when we save the value, it is equivalent to reading and writing the attribute value of UserKV directly, and also supports dynamically specifying the Key to assign the value. Compared with SharedPreferences, it has a great improvement in ease of use and readability, and completely shields the specific storage implementation logic for the external
/ / value
UserKV.name = "Yip Chi Chan"
UserKV.blog = "https://juejin.cn/user/923245496518439"
/ / value
val name = UserKV.name
val blog = UserKV.blog
// Dynamically specify the Key for assignment and value
UserKV.set("name"."Yip Chi Chan")
val name = UserKV.get("name"."")
Copy the codeHow it was designed
First, IKVHolder defines basic access methods that need to support custom data types in addition to basic data types. Thanks to Kotlin’s two syntactic features, extension and inline functions, we can access custom types without declaring generic types, which is very simple to use. JsonHolder implements the basic serialization and deserialization methods through Gson
interface IKVHolder {
companion object {
inline fun <reified T> IKVHolder.getBean(key: String): T {
return JsonHolder.toBean(get(key, ""))}inline fun <reified T> IKVHolder.getBeanOrNull(key: String): T? {
return JsonHolder.toBeanOrNull(get(key, ""))}inline fun <reified T> IKVHolder.getBeanOrDefault(key: String, defaultValue: T): T {
return JsonHolder.toBeanOrDefault(get(key, ""), defaultValue)
}
fun toJson(ob: Any?).: String {
return JsonHolder.toJson(ob)
}
}
// Data grouping, used to indicate the data cache in different ranges
val keyGroup: String
fun verifyBeforePut(key: String, value: Any?).: Boolean
fun get(key: String, default: Int): Int
fun set(key: String, value: Int)
fun <T> set(key: String, value: T?).
fun containsKey(key: String): Boolean
fun removeKey(vararg keys: String)
fun allKeyValue(a): Map<String, Any? >fun clear(a)...}Copy the codeBaseMMKVKVHolder implements the IKVHolder interface and internally introduces MMKV as a concrete persistent storage scheme
/ * * *@paramSelfGroup is used to specify data groups. Data in different groups are not associated with each other *@paramEncryptKey Specifies the encryption key. If empty, no encryption will be performed. */
sealed class BaseMMKVKVHolder constructor(
selfGroup: String,
encryptKey: String
) : IKVHolder {
final override val keyGroup: String = selfGroup
override fun verifyBeforePut(key: String, value: Any?).: Boolean {
return true
}
private val kv: MMKV? = if (encryptKey.isBlank()) MMKV.mmkvWithID(
keyGroup,
MMKV.MULTI_PROCESS_MODE
) else MMKV.mmkvWithID(keyGroup, MMKV.MULTI_PROCESS_MODE, encryptKey)
override fun set(key: String, value: Int) {
if(verifyBeforePut(key, value)) { kv? .putInt(key, value) } }override fun <T> set(key: String, value: T?). {
if (verifyBeforePut(key, value)) {
if (value == null) {
removeKey(key)
} else {
set(key, toJson(value))
}
}
}
override fun get(key: String, default: Int): Int {
returnkv? .getInt(key, default) ? : default }override fun containsKey(key: String): Boolean {
returnkv? .containsKey(key) ? :false
}
override fun removeKey(vararg keys: String){ kv? .removeValuesForKeys(keys) }override fun allKeyValue(a): Map<String, Any? > {valmap = mutableMapOf<String, Any? >() kv? .allKeys()? .forEach { map[it] = getObjectValue(kv, it) }return map
}
override fun clear(a){ kv? ClearAll ()}...}Copy the codeBaseMMKVKVHolder has two subclasses, the only difference is that MMKVKVFinalHolder stores key-value pairs that cannot be changed again. It is used to store data that cannot be changed twice and is only used for historical traceability, such as the timestamp, version number, version name of the application when it was first installed
/ * * *@paramSelfGroup is used to specify data groups. Data in different groups are not associated with each other *@paramEncryptKey Specifies the encryption key. If empty, no encryption will be performed. */
class MMKVKVHolder constructor(selfGroup: String, encryptKey: String = "") :
BaseMMKVKVHolder(selfGroup, encryptKey)
/** * The value cannot be changed twice after storage *@paramSelfGroup is used to specify data groups. Data in different groups are not associated with each other *@paramEncryptKey Specifies the encryption key. If empty, no encryption will be performed. */
class MMKVKVFinalHolder constructor(selfGroup: String, encryptKey: String = "") :
BaseMMKVKVHolder(selfGroup, encryptKey) {
override fun verifyBeforePut(key: String, value: Any?).: Boolean {
return! containsKey(key) } }Copy the codeThrough interface isolation, UserKV will not be exposed to the specific storage implementation mechanism, for developers just read and write UserKV property field, when we need to replace the storage solution, we only need to change the internal implementation of MMKVKVHolder. The upper level application doesn’t need any changes at all
KVHolder
The implementation of KVHolder is very simple, using Kotlin’s own powerful syntax features to further improve the ease of use and readability 馃槆馃槆 welcome to discuss
If you are interested, you can import dependencies directly from GitHub. Click here: KVHolder
allprojects {
repositories {
...
maven { url 'https://jitpack.io' }
}
}
dependencies {
implementation 'com.github.leavesC:KVHolder:latest_version'
}
Copy the code