preface
This article is the last article of MyBatis series, the previous two articles “MyBatis source parsing (a) – construction” and “MyBatis source parsing (ii) – implementation”, It mainly explains how MyBatis constructs our XML Configuration file into its internal Configuration object and MappedStatement object. Then in the second part, we explain how MyBatis executes our SQL statement step by step and encapsulates the result set. So this article as the last MyBatis series, naturally is to talk about MyBatis in a non-negligible function, level 1 cache and level 2 cache.
What is cache?
Although this article is about the cache of MyBatis, I hope that the friends who are learning computer can understand this article even if they have not used MyBatis framework.
What is ** cache? ** I’ll talk about my personal understanding, and finally the official concept.
Cache, “Cache,” as its name suggests, means temporary storage. Cache of the computer, we can understand directly to data stored in the memory of the container, this is discriminating and physical storage, because of the memory read/write speed is orders of magnitude higher than physical storage, so the program directly fetch the data from memory and the efficiency of the data from the physical hard disk is different, so there are often need to read the data, Designers often put them in a cache so that programs can read them. Cache is a price, however, so we just said, is cached in memory data in a container, a 64 gb of memory chips, it is usually possible to buy 3 or 4 pieces of 1 t – 2 t mechanical hard disk, so the cache cannot be used without restraint, such cost will increase, so the general data in the cache are frequently query, modify the data but not often.
In general services, the query usually goes through the following steps.
Read operation –> query cache already exists data –> if not, query database, if there is directly query cache –> database query returns data, at the same time, write into the cache.
Write operation –> clear cache data –> write database
More official concept:
▌ The Cache is the data exchange buffer (called the Cache). When a piece of hardware reads data, it first summarizes the data from the Cache, executes it directly, or retrives it from memory if it does not exist. Since cached data is much faster than memory, the cache's purpose is to help hardware run faster. ▌ The cache usually uses RAM (non-permanent storage after power failure), so files will be sent to a memory such as hard disk for permanent storage. The maximum cache on a computer is a memory stick, and there are 16 or 32 megabytes of cache on a hard drive. ☞ The cache is set up to mediate the difference in access speed between CPU and main memory. Generally, CPU speed is high, but memory speed is relatively low. To solve this problem, cache is usually used, and the cache access speed is between the CPU and main memory. The system will be some CPU in the recent several time period of frequent access to the content of the cache, so that to a certain extent to alleviate the main memory speed caused by the CPU "shutdown waiting for materials". ☞ A cache is simply storing data from external memory in memory. Why keep it in memory? Variables in all the programs we run are stored in memory, so if we want to put values in memory, we can store them as variables. Some caching in JAVA is generally implemented through a Map collection.Copy the code
MyBatis cache
Before saying MyBatis cache, first understand how the Java cache is generally implemented, we usually use the Map in Java to achieve cache, so the concept of cache in the later, it can be directly understood as a Map, store is the key value pair.
-
Introduction to Level 1 Caching
MyBatis level 1 cache is enabled by default and cannot be turned off. The default scope of level 1 cache is SqlSession. If SqlSession is built, the cache will exist. As long as the SqlSession is not closed, the same SQL processed by the SqlSession will not be called twice, and the cache will only be released when the session ends.
Although we cannot turn off level 1 caching, the scope can be changed, for example, to a Mapper.
Level 1 cache lifecycle:
1. If a SqlSession calls the close() method, the Tier 1 cache PerpetualCache object is released and tier 1 cache is not available.
2. If a SqlSession calls clearCache(), the data in the PerpetualCache object is cleared, but the object is still usable.
3. Every UPDATE operation performed in SqlSession (update(), delete(), INSERT ()) clears data from the PerpetualCache object, but the object continues to be used.
Adapted from: www.cnblogs.com/happyflying…
-
Introduction to Level 2 Cache
MyBatis level 2 cache is disabled by default. There are two ways to enable it:
-
Add the following configuration fragment to mybatis-config.xml
<! -- Global configuration parameters, set as needed --> <settings> <! -- Enable level 2 cache. Default value: true --> <setting name="cacheEnabled" value="true"/> </settings> Copy the code -
Enable it in mapper. XML
<! -- Enable level 2 cache in namespace of mapper --> <! MyBatis is designed to exclude cache reclamation policies from any design design. (1) LRU, the least recently used object, and the most recently used object. (2) FIFO, which removes objects in the order in which they enter the cache. (3) SOFT, which removes objects based on garbage collector status and SOFT reference rules. More aggressively remove objects based on garbage collector state and weak reference rules. FlushInterval: specifies the flushInterval in milliseconds. If you do not configure it, the cache will be flushed only when the SQL is executed. Size: indicates the number of references. It is a positive integer. It indicates the maximum number of objects that can be stored in the cache. Too many Settings will cause memory overflow. ReadOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects readOnly: 1024 objects <cache eviction="Recycling Strategy" type="Cache class"/> Copy the code
Level 2 cache has a different scope than level 1 cache. Level 1 cache has a SqlSession, but level 2 cache has a namespace. All SQL sessions operating in the Mapper can share this level 2 cache. But if you have two identical SQL statements written to different namespaces, the SQL will be executed twice, producing two caches with the same value.
-
Execution flow of MyBatis cache
Again using the test cases from the previous two articles, let’s look at how caching is performed from a source code perspective.
public static void main(String[] args) throws Exception {
String resource = "mybatis.xml";
InputStream inputStream = Resources.getResourceAsStream(resource);
SqlSessionFactory sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
SqlSession sqlSession = sqlSessionFactory.openSession();
// SqlSession is the object that deals with the database from the caller's point of view
DemoMapper mapper = sqlSession.getMapper(DemoMapper.class);
Map<String,Object> map = new HashMap<>();
map.put("id"."2121");
// Executing this method actually goes to invoke
System.out.println(mapper.selectAll(map));
sqlSession.close();
sqlSession.commit();
}
Copy the codeThe query() method is executed here:
public <E> List<E> query(MappedStatement ms, Object parameterObject, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql)
throws SQLException {
// The level 2 Cache is obtained from MappedStatement
Cache cache = ms.getCache();
if(cache ! =null) {
// Whether the cache needs to be flushed
// In the
flushCacheIfRequired(ms);
// Check whether the mapper has level 2 caching enabled
if (ms.isUseCache() && resultHandler == null) {
/ / regardless of
ensureNoOutParams(ms, boundSql);
@SuppressWarnings("unchecked")
// Get it from cache first
List<E> list = (List<E>) tcm.getObject(cache, key);
if (list == null) {
// If cache is empty, query level 1 cache
list = delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
// Put the data into the level 2 cache after the query
tcm.putObject(cache, key, list); // issue #578 and #116
}
/ / return
returnlist; }}// If the level 2 cache is null, the level 1 cache is queried directly
return delegate.query(ms, parameterObject, rowBounds, resultHandler, key, boundSql);
}
Copy the codeIt can be seen that MyBatis will first check whether the mapper has enabled level 2 cache when querying data. If so, it will query level 2 cache first. If there is the data we need in the cache, we will directly return the data from the cache.
If the second level cache does not exist, does it query the data directly? The answer is no, if level 2 cache does not exist, MyBatis will query level 1 cache again and look further.
public <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey key, BoundSql boundSql) throws SQLException {
ErrorContext.instance().resource(ms.getResource()).activity("executing a query").object(ms.getId());
if (closed) {
throw new ExecutorException("Executor was closed.");
}
if (queryStack == 0 && ms.isFlushCacheRequired()) {
clearLocalCache();
}
List<E> list;
try {
queryStack++;
// Query level 1 cache (localCache)
list = resultHandler == null ? (List<E>) localCache.getObject(key) : null;
if(list ! =null) {
// There is output resource processing for the stored procedure
handleLocallyCachedOutputParameters(ms, key, parameter, boundSql);
} else {
// If the cache is empty, fetch from the database
list = queryFromDatabase(ms, parameter, rowBounds, resultHandler, key, boundSql);
// Put a placeholder localCache.putobject (key, EXECUTION_PLACEHOLDER) into the cache; try { list = doQuery(ms, parameter, rowBounds, resultHandler, boundSql); } finally { localCache.removeObject(key); } // Put the real data to the level cache localCache.putobject (key, list); if (ms.getStatementType() == StatementType.CALLABLE) { localOutputParameterCache.putObject(key, parameter); } return list; * /}}finally {
queryStack--;
}
if (queryStack == 0) {
for (DeferredLoad deferredLoad : deferredLoads) {
deferredLoad.load();
}
// issue #601
deferredLoads.clear();
if (configuration.getLocalCacheScope() == LocalCacheScope.STATEMENT) {
// issue #482clearLocalCache(); }}return list;
}
Copy the codeLevel 1 caches and level 2 caches have similar query logic. If there is no result in level 1 caches, then the database is queried directly and then the level 2 caches are written back.
In fact, the level 1 cache and level 2 cache execution process is finished, the cache logic is actually similar, MyBatis cache is first query level 1 cache and then query level 2 cache.
But that’s not the end of the article, and there are some cache-related issues to talk about.
Cache transaction problem
I don’t know if you’ve ever thought about this, but let’s say we have a scenario where we use level 2 caching, because level 2 caching is cross-transaction.
Suppose we start the transaction before the query and do the database operation:
1. INSERT data into database (INSERT)
2. Query data within the same transaction (SELECT)
3. COMMIT transaction
4. ROLLBACK failed
Let’s analyze this scenario, the first SqlSession perform an INSERT operation, obviously, in which we have just on the basis of analysis of logic, the cache will be cleared, and then in the same transaction data query, data is loaded from the database again to the cache, at this time to commit the transaction, and transaction commit failed. Consider what would happen at this time, I believe that has been thought of, after the failure of transaction commit, the transaction will be rolled back, then executes the INSERT in the data will be rolled back, but we made a query after the insertion, the data already in the cache, the next query is necessarily query cache and won’t go to query the database directly, However, there are already data inconsistencies between the cache and the database.
The root cause of the problem is that a failed database commit transaction can be rolled back, but the cache cannot.
Let’s see how MyBatis solves this problem.
-
TransactionalCacheManager
This class is MyBatis for caching transaction management, we can take a look at its data structure.
public class TransactionalCacheManager { // Transaction cache private final Map<Cache, TransactionalCache> transactionalCaches = new HashMap<>(); public void clear(Cache cache) { getTransactionalCache(cache).clear(); } public Object getObject(Cache cache, CacheKey key) { return getTransactionalCache(cache).getObject(key); } public void putObject(Cache cache, CacheKey key, Object value) { getTransactionalCache(cache).putObject(key, value); } public void commit(a) { for(TransactionalCache txCache : transactionalCaches.values()) { txCache.commit(); }}public void rollback(a) { for(TransactionalCache txCache : transactionalCaches.values()) { txCache.rollback(); }}private TransactionalCache getTransactionalCache(Cache cache) { return transactionalCaches.computeIfAbsent(cache, TransactionalCache::new); }}Copy the codeTransactionalCacheManager encapsulates a Map for the transaction cache object caching, the Map Key is our second level cache object, and Value is called a TransactionalCache, as the name implies, this cache cache is affairs, Let’s look at its internal implementation.
public class TransactionalCache implements Cache { private static final Log log = LogFactory.getLog(TransactionalCache.class); // Real cache object private final Cache delegate; // Whether to clear the identity of the submission space private boolean clearOnCommit; // All caches to commit private final Map<Object, Object> entriesToAddOnCommit; // Set of missed caches to prevent breakdown of caches, and if the queried data is null, it indicates that there may be data inconsistency through the database, and all records are recorded in this place private final Set<Object> entriesMissedInCache; public TransactionalCache(Cache delegate) { this.delegate = delegate; this.clearOnCommit = false; this.entriesToAddOnCommit = new HashMap<>(); this.entriesMissedInCache = new HashSet<>(); } @Override public String getId(a) { return delegate.getId(); } @Override public int getSize(a) { return delegate.getSize(); } @Override public Object getObject(Object key) { // issue #116 Object object = delegate.getObject(key); if (object == null) { // If it is empty, put it in the missed cache, and putObject puts the key-value pairs that should have been in the real cache into the commit transaction cache after querying the database entriesMissedInCache.add(key); } // If not empty // issue #146 // Check whether the cache clearing flag is false, or true if the transaction commits, which updates the cache, so null is returned. if (clearOnCommit) { return null; } else { // If the transaction is not committed, the data in the original cache is returned, returnobject; }}@Override public void putObject(Object key, Object object) { // If the data returned is null, it is possible to query the database. The queried data is placed in the cache of the transaction to be committed // It should be put into the cache, but now it is put into the cache of the transaction to be committed. entriesToAddOnCommit.put(key, object); } @Override public Object removeObject(Object key) { return null; } @Override public void clear(a) { // If the transaction commits, set the clear cache commit flag to true clearOnCommit = true; / / empty entriesToAddOnCommit entriesToAddOnCommit.clear(); } public void commit(a) { if (clearOnCommit) { // If true, the cache is cleared. delegate.clear(); } // Flush local cache to real cache. flushPendingEntries(); // Then reset all values. reset(); } public void rollback(a) { // Transaction rollback unlockMissedEntries(); reset(); } private void reset(a) { // Reset operation. clearOnCommit = false; entriesToAddOnCommit.clear(); entriesMissedInCache.clear(); } private void flushPendingEntries(a) { // Iterate over the cache to commit in the transaction manager for (Map.Entry<Object, Object> entry : entriesToAddOnCommit.entrySet()) { // Write to the real cache. delegate.putObject(entry.getKey(), entry.getValue()); } for (Object entry : entriesMissedInCache) { // Put the missed ones together if(! entriesToAddOnCommit.containsKey(entry)) { delegate.putObject(entry,null); }}}private void unlockMissedEntries(a) { for (Object entry : entriesMissedInCache) { // Clear missed caches in the real cache. try { delegate.removeObject(entry); } catch (Exception e) { log.warn("Unexpected exception while notifiying a rollback to the cache adapter." + "Consider upgrading your cache adapter to the latest version. Cause: "+ e); }}}}Copy the codeIn TransactionalCache there is a real Cache object Cache, which is our real secondary Cache, and entriesToAddOnCommit, which is a Map object that holds the Cache of all pending transactions.
We saw in the second level cache implementation code, a get or put the results in the cache, is called the object of TCM call getObject () method and putObject () method, the object is actually TransactionalCacheManager entity object, This object actually calls TransactionalCache methods. Let’s look at how these two methods are implemented.
@Override public Object getObject(Object key) { // issue #116 Object object = delegate.getObject(key); if (object == null) { // If it is empty, put it in the missed cache, and putObject puts the key-value pairs that should have been in the real cache into the commit transaction cache after querying the database entriesMissedInCache.add(key); } // If not empty // issue #146 // Check whether the cache clearing flag is false, or true if the transaction commits, which updates the cache, so null is returned. if (clearOnCommit) { return null; } else { // If the transaction is not committed, the data in the original cache is returned, returnobject; }}@Override public void putObject(Object key, Object object) { // If the data returned is null, it is possible to query the database. The queried data is placed in the cache of the transaction to be committed // It should be put into the cache, but now it is put into the cache of the transaction to be committed. entriesToAddOnCommit.put(key, object); } Copy the codeThere are two branches in the getObject() method:
If null is found in the cache, the key will be placed in the entriesMissedInCache. The main purpose of this object is to save all the missed keys, so that the cache will not be broken. In addition, if we cannot find the data in the cache, the key will be stored in the entriesMissedInCache. Then it is possible to query in level 1 caches and databases, after which the putObject() method is called. This method is supposed to put the data we queried into the real cache, but for now because of the transaction, it will be put into entriesToAddOnCommit.
If the value of clearOnCommit is true, the transaction has been committed and the cache will be cleared, so null is returned. If false, the transaction has not yet been committed. So returns the data currently stored in the cache.
What happens when the transaction commits successfully or fails? ** Look at the COMMIT and rollback methods.
public void commit(a) { if (clearOnCommit) { // If true, the cache is cleared. delegate.clear(); } // Flush local cache to real cache. flushPendingEntries(); // Then reset all values. reset(); } public void rollback(a) { // Transaction rollback unlockMissedEntries(); reset(); } Copy the codeIf the transaction commits successfully, the following steps will be performed:
- Clear the real cache.
- Flush the local cache (uncommitted transaction cache entriesToAddOnCommit) to the real cache.
- Reset all values.
Let’s look at how the code is implemented:
private void flushPendingEntries(a) { // Iterate over the cache to commit in the transaction manager for (Map.Entry<Object, Object> entry : entriesToAddOnCommit.entrySet()) { // Write to the real cache. delegate.putObject(entry.getKey(), entry.getValue()); } for (Object entry : entriesMissedInCache) { // Put the missed ones together if(! entriesToAddOnCommit.containsKey(entry)) { delegate.putObject(entry,null); }}}private void reset(a) { // Reset operation. clearOnCommit = false; entriesToAddOnCommit.clear(); entriesMissedInCache.clear(); } public void clear(a) { // If the transaction commits, set the clear cache commit flag to true clearOnCommit = true; // Clear the transaction commit cache entriesToAddOnCommit.clear(); } Copy the codeTo clear the real cache, Map calls the clear method to clear all key-value pairs.
To flush the uncommitted transaction cache to the real cache, entriesToAddOnCommit is iterated, and the putObject method of the real cache is called to put the key value pairs in entriesToAddOnCommit into the real cache. Once this is done, Missing cache data is also put in with a value of NULL.
Finally, reset, set the committed transaction identifier to false, and clear all data in the cache of missed and uncommitted transactions.
If the transaction is not committed properly, then a rollback will occur. Let’s look at the rollback process:
- Clear missed caches in the real cache.
- Reset all values
public void rollback(a) { // Transaction rollback unlockMissedEntries(); reset(); } private void unlockMissedEntries(a) { for (Object entry : entriesMissedInCache) { // Clear missed caches in the real cache. try { delegate.removeObject(entry); } catch (Exception e) { log.warn("Unexpected exception while notifiying a rollback to the cache adapter." + "Consider upgrading your cache adapter to the latest version. Cause: "+ e); }}}Copy the codeAny missing key in the cache will be recorded in the entriesMissedInCache, so this cache contains all the query database keys, so you only need to delete this part of the key and the corresponding value in the real cache.
-
Cache transaction summary
In short, cache transaction control mainly through TransactionalCacheManager TransactionCache, The key lies in the TransactionCache objects entriesToAddCommit and entriesMissedInCache. EntriesToAddCommit acts as a substitute for the real cache between the start of a transaction and the commit. The data queried from the database will be put into the Map first, and the data in the object will be refreshed to the real cache after the transaction submission. If the transaction submission fails, the data in the cache will be cleared, and the real cache will not be affected.
EntriesMissedInCache is used to store the key that is not hit in the cache during the query. If the key is not hit, it needs to be queried in the database. Then the query will be saved in entriesToAddCommit. Rollback will use the keys stored in the entriesMissedInCache to clean up the real cache. This will ensure that the data cached in the transaction is consistent with the data in the database.
Some experience with caching
-
The level 2 cache cannot contain ever-increasing data
The level 2 cache is not a SqlSession but a namespace, so the level 2 cache will exist when your application is started until the application is shut down. Therefore, the level 2 cache cannot contain more and more data over time, which may cause memory space to be used up.
-
Level 2 cache may have dirty reads (avoidable)
Since the scope of the level 2 cache is namespace, we can assume a scenario where two namespaces operate on a table, the first namespace queries the table and writes it back to memory, and the second namespace inserts a data entry into the table. The second level cache of the first namespace will not clear the contents of the cache. In the next query, the cache will be used to query, which will cause data inconsistency.
Therefore, it is best not to use level 2 caching when there are multiple namespaces operating on the same table in the project, or to avoid using two namespaces operating on the same table when using level 2 caching.
-
Spring implements MyBatis cache invalidation
The level 1 cache scope is SqlSession, and the user can define when the SqlSession occurs and when it is destroyed. Level 1 cache exists during this period. When the consumer calls the close() method, the level 1 cache is destroyed.
However, after we integrate with Spring, Spring has skipped SqlSessionFactory and we can call Mapper directly. As a result, Spring destroys SqlSession and level 1 cache after the database operation. So the level 1 cache is invalidated.
So how do you make caching work?
1. Start a transaction, because once a transaction is started, Spring does not destroy the SqlSession after executing the SQL, because once SqlSession is closed, the transaction is gone, and once we start the transaction, the cache will remain for the duration of the transaction.
2. Use level 2 cache.
conclusion
Hello world.
Welcome to visit my personal Blog: Object’s Blog