The problem background
Have students feedback, in their own business calls groovy scripts dynamically generated some class, the class can’t discharge phenomenon, below XElephant from fake you stupid great god PerfMa companies “memory. The console. Heapdumps. Cn/”
If you want to do offline analysis, you can also use JProfile (paid), YourKit, etc.
You can see that there are 4808 classLoaders. The total number of classes loaded by these classLoaders is 9612. Loaded classes one is defined in our groovy com. Yuping. App214c2d6e_8f0e_209a_7cbf_81130c799181. BookDataModel class.
None of these classes can be unloaded by GC. The startup parameters are as follows:
java -Xmx2688M -Xms2688M -Xmn960M -XX:MaxMetaspaceSize=512M -XX:MetaspaceSize=512M -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSClassUnloadingEnabled -XX:+ParallelRefProcEnabled -XX:+CMSScavengeBeforeRemark -XX:ErrorFile=/tmp/hs_err_pid%p.log -Xloggc:/tmp/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -verbose:class -XX:+PrintClassHistogramBeforeFullGC -XX:+PrintClassHistogramAfterFullGC -XX:+PrintCommandLineFlags -XX:+PrintHeapAtGC -XX:-DisableExplicitGC -jar The target/groovy - demo - project - 1.0 - the SNAPSHOT. The jarCopy the codeThis parameter allows CMS classes to be uninstalled.
The business logic
The basic logic is to load a Groovy script dynamically from db
@Service
public class MyService {
@Resource
private MongoTemplate mongoTemplate;
void insert(a) {
GroovyClassLoader groovyClassLoader = null;
try {
groovyClassLoader = new GroovyClassLoader();
Class<? extends BaseClazz> dataModelClazz = groovyClassLoader.parseClass("groovy content");
// The real business is that the data is passed in from the outside, the data structure has the data, here simplifies processing
JSONObject data = new JSONObject();
data.put("id", UUID.randomUUID().toString());
data.put("enterpriseCode"."foo");
BaseClazz model = JSON.toJavaObject(data, dataModelClazz);
BaseClazz newModel = mongoTemplate.insert(model, "test_ya");
} catch (Exception e) {
e.printStackTrace();
} finally{ groovyClassLoader.clearCache(); }}}Copy the codeThe groovy script looks like this, with a simple subclass definition:
package com.yuping.app214c2d6e_8f0e_209a_7cbf_81130c799181 import com.imdach.demo.BaseClazz class bookDataModel extends BaseClazz { String author String charter // ... Omit many fields and methods}Copy the codeThe first thing I thought about when I got this question was what the conditions for class uninstallation were.
-
First of all, the first requirement is “all instances of this class are unreachable and GC”, otherwise the instance is still there, the class is gone, just like people have no soul, it is impossible.
-
The second requirement is that the ClassLoader of the Class is unreachable and GC is performed. It is understood that the ClassLoader needs to hold a reference to the Class, otherwise it cannot determine whether a Class has been loaded or not and cannot implement the basic functions of Class loading.
-
The third requirement, which is not referenced by any other GC Root, is understandable. This applies to all scenarios. Reachable objects should not be reclaimed.
-
Fourth requirement: Triggering GC (FullGC). The scenario of class unloading is relatively rare. Taking CMS as an example, class unloading is triggered at FullGC.
The first condition is that the class instance is unreachable, which is obvious because the class instance is a local variable and the function is unreachable after the call.
The second condition is ClassLoader unreachable, which is OK in this scenario. Every time a Groovy script is loaded, it is a new ClassLoader, which is ready to be GC.
The third condition is not referenced by any other GC Root.
The fourth condition, trigger GC (FullGC), which can also be excluded, has already been triggered manually and would have been triggered once when dumping heap memory.
So the next step is to see if this class is referenced by GC Root.
The object is referenced by
We find one of these classes, like the first one, whose address is 0x79357f308
Next, switch to the “Object View” screen and find the object by its address to find more detailed information about the object.
First of all, I saw the Fastjson library, which is how to get involved in the goods, I don’t just call you this tool to do a serialization?
Can see com. Yuping. App214c2d6e_8f0e_209a_7cbf_81130c799181. BookDataModel was com. Alibaba. Fastjson. Util. IdentityHashMap $Entry The bookDataModel class is referenced, as you might guess by name, in a Fastjson hashMap.
Why is it in a HashMap? Let’s see what it does. IdentityHashMap (ParserConfig, SerializeConfig, ParserConfig, global) All subsequent calls use static variables, which are not GC.
public class ParserConfig {
public static ParserConfig getGlobalInstance(a) {
return global;
}
public static ParserConfig global = new ParserConfig();
private final IdentityHashMap<Type, ObjectDeserializer> deserializers = new IdentityHashMap<Type, ObjectDeserializer>();
Copy the codeFastJson analytical process, the com. Yuping. App214c2d6e_8f0e_209a_7cbf_81130c799181. BookDataModel class in IdentityHashMap, that’s cool, The global GC Root holds deserialmap in the IdentityHashMap where bookDataModel class is stored.
At this point, we can take care of the FastJson problem. I looked it up. It has a manual cleanup function
public class ParserConfig {
public void clearDeserializers(a) {
this.deserializers.clear();
this.initDeserializers(); }}Copy the codeThat way I can empty the HashMap so that I don’t hold any references to that class.
@Service
public class MyService {
@Resource
private MongoTemplate mongoTemplate;
void insert(a) {
GroovyClassLoader groovyClassLoader = null;
try {
/ / to omit
} catch (Exception e) {
e.printStackTrace();
} finally {
groovyClassLoader.clearCache();
// Add the following two linesSerializeConfig.getGlobalInstance().clearSerializers(); ParserConfig.getGlobalInstance().clearDeserializers(); }}}Copy the codeI thought the problem was solved, relieved to let the development students to change, and then wait to say “problem solved”, the result said, class or not uninstall, slap slap face.
World War II unloading
The class is still being referenced by other objects, but this time there is no FastJson, this time there is a lot of Spring related information.
Can see bookDataModel class is org. Springframework. Data. The util. ClassTypeInformation object reference type field, ClassTypeInformation classes are defined as follows.
public class ClassTypeInformation<S> extends TypeDiscoverer<S> {
private final Class<S> type;
}
Copy the codeThe type field here stores the bookDataModel class generated by groovy.
On one of the org. Springframework. Data. Util. ClassTypeInformation, looked into the upper chain of GC.
You can see that the ClassTypeInformation object is referenced by the persistentEntities field of the MongoMappingContext object.
public abstract class AbstractMappingContext {
private finalMap<TypeInformation<? >, Optional<E>> persistentEntities =new HashMap<>();
}
public class MongoMappingContext extends AbstractMappingContext {}Copy the codeBecause MongoMappingContext is a long-standing Spring singleton Bean, persistentEntities will not be GC and will refer to ClassTypeInformation, ClassTypeInformation refers to the bookDataModel class, causing the bookDataModel class to be unable to be reclaimed.
At this point, it becomes clear why. As to solve, so that I don’t know much about, need to be familiar with the spring – directing a classmate see how to bypass the cache mechanism, in the spring to customize a AbstractMongoConfiguration, let the spring not cache (I won’t).
Here I have a very immature solution, directly using bare mongodb-java-driver, tested OK, but not recommended.
@Service
public class MyService {
@Resource
private MongoTemplate mongoTemplate;
void insert(a) {
GroovyClassLoader groovyClassLoader = null;
try {
groovyClassLoader = new GroovyClassLoader();
File f = new File("test.groovy");
Class<? extends BaseClazz> dataModelClazz = groovyClassLoader.parseClass(FileUtils.readFileToString(f));
JSONObject data = new JSONObject();
data.put("id", UUID.randomUUID().toString());
data.put("enterpriseCode"."foo");
BaseClazz model = JSON.toJavaObject(data, dataModelClazz);
CodecRegistry pojoCodecRegistry = fromProviders(PojoCodecProvider.builder().automatic(true).build());
CodecRegistry codecRegistry = fromRegistries(MongoClientSettings.getDefaultCodecRegistry(), pojoCodecRegistry);
MongoClientSettings clientSettings = MongoClientSettings.builder()
.applyConnectionString(new ConnectionString("mongodb://localhost:27017"))
.codecRegistry(codecRegistry)
.build();
try (MongoClient mongoClient = MongoClients.create(clientSettings)) {
MongoDatabase mongoDatabase = mongoClient.getDatabase("seewo_easi_pass");
MongoCollection collection = mongoDatabase.getCollection("test_ya", dataModelClazz); InsertOneResult result = collection.insertOne(model); System.out.println(result); }}catch (Exception e) {
e.printStackTrace();
} finally{ groovyClassLoader.clearCache(); SerializeConfig.getGlobalInstance().clearSerializers(); ParserConfig.getGlobalInstance().clearDeserializers(); }}}Copy the codeAfter the experiment, it is true that the class can be uninstalled after GC, but the related class can not be found through the memory dump check.
summary
FastJson IdentityHashMap FastJson IdentityHashMap FastJson IdentityHashMap FastJson IdentityHashMap As for MongoDB this is really not expected to encounter, probably the author did not think, there will be someone dynamically generated classes and the corresponding class instance, and then insert MongoDB bar.
Can reproduce the problem, in fact, is not a problem, the solution is only a matter of time. The above solutions may be wrong, just look at the ideas.