Json-lib

Json-lib is a popular Json library for Java. The latest version is 2.4, with JDK 1.3 and 1.5 support, respectively. It was updated on December 14, 2010. Although it has not been maintained for many years, a search engine search for “Java Json” and related keywords shows that the library is still being introduced and used. The project’s official website is json-lib.sourceforge.net/.

【 conclusion 】

Json-lib will parse each Json object through a string and substring the current parsing position to the end of the string. So when you encounter large Json data with many objects, you do a lot of character array copying, which leads to a lot of CPU and memory consumption, and even serious Full GC problems.

【 答 案 】

One day, it was found that there were a lot of Full GC problems in the online production server. The investigation found that the amount of an old interface would increase when Full GC was generated, but this interface only parses Json but stores the parsed data in the cache. Therefore, it was suspected that it was related to the size of the interface request parameters. The log shows that there is much larger Json data than the average request, but only about 1MB. To simplify this problem, write the following performance test code.


package net.mayswind;

import net.sf.json.JSONObject;
import org.apache.commons.io.FileUtils;

import java.io.File;


public class JsonLibBenchmark {
    public static void main(String[] args) throws Exception {
        String data = FileUtils.readFileToString(new File("Z:\\data.json"));
        benchmark(data, 5);
    }

    private static void benchmark(String data, int count) {
        long startTime = System.currentTimeMillis();

        for (int i = 0; i < count; i++) {
            JSONObject root = JSONObject.fromObject(data);
        }

        long elapsedTime = System.currentTimeMillis() - startTime;
        System.out.println(String.format("count=%d, elapsed time=%d ms, avg cost=%f ms", count, elapsedTime, (double) elapsedTime / count)); }}Copy the code

The above code takes an average of about 7 seconds per parse after execution, as shown in the figure below.



Json file for testing, “…” The whole Json data contains more than 30,000 Json objects. The actual test data is shown in the figure below.

{
    "data": [{"foo": 0123456789,
            "bar": 1234567890}, {"foo": 0123456789,
            "bar": 1234567890},... ] }Copy the code



Using Java Mission Control to record the execution, as shown in the figure below, you can see that a large number of char[] arrays are allocated.



The main content of jsonObject._fromJsonTokener method is shown below. You can see that it matches whether it starts with “null” at the beginning of the code.

private static JSONObject _fromJSONTokener(JSONTokener tokener, JsonConfig jsonConfig) {
    try {
        if (tokener.matches("null.*")) {
            fireObjectStartEvent(jsonConfig);
            fireObjectEndEvent(jsonConfig);
            return new JSONObject(true);
        } else if(tokener.nextClean() ! ='{') {
            throw tokener.syntaxError("A JSONObject text must begin with '{'");
        } else{ fireObjectStartEvent(jsonConfig); Collection exclusions = jsonConfig.getMergedExcludes(); PropertyFilter jsonPropertyFilter = jsonConfig.getJsonPropertyFilter(); JSONObject jsonObject = new JSONObject(); .Copy the code

The matches method simply uses a substring to capture the string at the end of the current position and then performs a regular match.

public boolean matches(String pattern) {
    String str = this.mySource.substring(this.myIndex);
    return RegexpUtils.getMatcher(pattern).matches(str);
}Copy the code

The String substring creates a new String by passing in an array of characters, a starting position, and a truncated length.

public String substring(int beginIndex) {
    if (beginIndex < 0) {
        throw new StringIndexOutOfBoundsException(beginIndex);
    }
    int subLen = value.length - beginIndex;
    if (subLen < 0) {
        throw new StringIndexOutOfBoundsException(subLen);
    }
    return (beginIndex == 0) ? this : new String(value, beginIndex, subLen);
}Copy the code

In JDK7 and above, this constructor is called with a copy of the truncated data in the last line, which is where the whole problem lies.

public String(char value[], int offset, int count) {
    if (offset < 0) {
        throw new StringIndexOutOfBoundsException(offset);
    }
    if (count <= 0) {
        if (count < 0) {
            throw new StringIndexOutOfBoundsException(count);
        }
        if (offset <= value.length) {
            this.value = "".value;
            return;
        }
    }
    // Note: offset or count might be near -1>>>1.
    if (offset > value.length - count) {
        throw new StringIndexOutOfBoundsException(offset + count);
    }
    this.value = Arrays.copyOfRange(value, offset, offset+count);
}Copy the code

BLOG address: www.liangsonghua.com

Pay attention to wechat public number: songhua preserved egg bulletin board, get more exciting!

Introduction to our official account: We share our technical insights from working in JD, as well as JAVA technology and best practices in the industry, most of which are pragmatic, understandable and reproducible