preface

I haven’t written a blog for a long time. It’s been 2 months since I started my new employer.

Blogging has to pick up again. It takes at least 21 days to build a good habit but all it takes is an excuse to ruin it

describe

I have done a task recently, mainly dealing with data. All the original data is stored in our Nosql database, and WE use AWS DynamoDB. Because the data in each case cannot be delivered to the other party completely, we can only deliver it selectively!

Analysis of the

First of all, we often see several online processing methods, really online articles are really copy copy or copy, looking for a long time did not find the right method, if it is simple processing:

  • You can annotate JSONField using FastJson annotations and set serialize =fasle
  • Or use the Java keyword TRANSIENT

However, I can’t use the above 2. The entities of scheme of each table structure are already encapsulated in the table. I also don’t want to duplicate the wheel!

The use of SimplePropertyPreFilter

SimplePropertyPreFilter (SimplePropertyPreFilter) ¶ SimplePropertyPreFilter (SimplePropertyPreFilter)

But using SimplePropertyPreFilter doesn’t work for me. Why?

There are two problems with using SimplePropertyPreFilter

Filter fields with duplicate names

It may be more abstract, so LET me write a demo and I believe that a small partner can understand:

{
	"contacts": [{
		"age": 30."linkPhone": "05148891922"."name": "burgxun"
	}, {
		"age": 30."linkPhone": "199888888"."name": "And"}]."orderAmount": 890.23."orderNO": "BU0000452"."name": "This is the name of a special tag."."resource": {
		"name": "Suzhou Two Flights to Japan"."producer": "Same Journey CITS"."type": 1}}Copy the code

For example, in the JSON above, I want to remove the name;

 SimplePropertyPreFilter simplePropertyPreFilter = new SimplePropertyPreFilter();
        simplePropertyPreFilter.getExcludes().add("name");
        String jsonString = JSON.toJSONString(order, simplePropertyPreFilter);
        System.out.println(jsonString);
Copy the code

Results of execution:

{
	"contacts": [{
		"age": 30."linkPhone": "05148891922"
	}, {
		"age": 30."linkPhone": "199888888"}]."orderAmount": 890.23."orderNO": "BU0000452"."resource": {
		"producer": "Same Journey CITS"."type": 1}}Copy the code

So obviously you see the problem and you take all the names out of it

Improved version

There is a way to improve it;

 SimplePropertyPreFilter simplePropertyPreFilter = new SimplePropertyPreFilter(TravelOrder.class);
        simplePropertyPreFilter.getExcludes().add("name");
        String jsonString = JSON.toJSONString(order, simplePropertyPreFilter);
        System.out.println(jsonString);
Copy the code

Results of execution:

{
	"contacts": [{
		"age": 30."linkPhone": "05148891922"."name": "burgxun"
	}, {
		"age": 30."linkPhone": "199888888"."name": "And"}]."orderAmount": 890.23."orderNO": "BU0000452"."resource": {
		"name": "Suzhou Two Flights to Japan"."producer": "Same Journey CITS"."type": 1}}Copy the code

This modified version specifies that the corresponding filter is used on a class

Still inflexible

This eliminates the problem of duplicate filtering, but if I need to filter the producer (Resource), linkPhone (Contacts), and orderAmount (contacts) fields, I need to filter the producer (Resource), linkPhone (Contacts), and orderAmount fields.

SimplePropertyPreFilter (SimplePropertyPreFilter, SimplePropertyPreFilter, SimplePropertyPreFilter);

Why am I mentioning the class above, because WHEN I did it, I made a big detour, I could get the outermost entity of the big object, but when I was filtering inside, if this was not the base type, if I wrote a SimplePropertyPreFilter, The class is very difficult to get. I get the entity type in the field by reflection! This also has to go through a section of field search type judgment and access and so on ~

Many people may not understand, but let’s give an example:

class TravelOrder {
    private String orderNO;
    private Float orderAmount;
    private String name;
    private Resource resource;
    privateList<Passenger> contacts; . }class Resource {
    private String name;
    private String producer;
    privateInteger type; . }class Passenger {
    private String name;
    private String linkPhone;
    privateInteger age; . }Copy the code

As I mentioned above, I want to filter the producer field inside the Resource field. In this case, I only pass in the Class of the TravelOrder field. In this case, I need to reflect the type of the resource field to get the type. Then write a SimplePropertyPreFilter, and maybe a Resource. Class. So if we do this, we’re going to have to do a lot of these filters and if the fields of the SimplePropertyPreFilter are different for each class, we’re going to have to do a lot of repetitive work and the code is not flexible

The final version

So with that in mind, let’s take a look at how SimplePropertyPreFilter is implemented and can we customize one ourselves

public class SimplePropertyPreFilter implements PropertyPreFilter {
    private finalClass<? > clazz;// The target class of the filter is null by default if it is not passed
    private final Set<String> includes;// The field to render
    private final Set<String> excludes;// Fields that do not need to be rendered
    private int maxLevel;// The filter hierarchy is our JSON hierarchy. You can test it yourself
    public SimplePropertyPreFilter(String... properties) {
        this((Class)null, properties);
    }
    // Return true if it can render false if it cannot
    public boolean apply(JSONSerializer serializer, Object source, String name) {
        if (source == null) {
            return true;
        } else if (this.clazz ! =null&&!this.clazz.isInstance(source)) {
            return true;If clazz is not null, the type to be used is specified. If isInstance is false, the filter should not be valid for it, so return true
        } else if (this.excludes.contains(name)) {
            return false;// Focus on the side and the following includes
        } else {
            if (this.maxLevel > 0) {
                int level = 0;

                for(SerialContext context = serializer.context; context ! =null; context = context.parent) {
                    ++level;
                    if (level > this.maxLevel) {
                        return false; }}}return this.includes.size() == 0 || this.includes.contains(name); }}}Copy the code

Look at the above code we know that the main is to use the name passed to do the judgment;

Let’s test the code:

 SimplePropertyPreFilter simplePropertyPreFilter = new SimplePropertyPreFilter();
        simplePropertyPreFilter.getExcludes().add("orderAmount");
        simplePropertyPreFilter.getExcludes().add("resource.producer");
        simplePropertyPreFilter.getExcludes().add("contacts.linkPhone");
        String jsonString = JSON.toJSONString(order, simplePropertyPreFilter);
        System.out.println(jsonString);
Copy the code

Take a look at the results:

{
	"contacts": [{
		"age": 30."linkPhone": "05148891922"."name": "burgxun"
	}, {
		"age": 30."linkPhone": "199888888"."name": "And"}]."name": "This is the name of a special tag."."orderNO": "BU0000452"."resource": {
		"name": "Suzhou Two Flights to Japan"."producer": "Same Journey CITS"."type": 1}}Copy the code

The filter resource. Producer, contacts.linkPhone does not work. Only the outermost orderAmount is filtered.

In that case, let’s change the methods in Apply

Filter with hierarchical property PropertyPreFilter

class MyPropertyPreFilter implements PropertyPreFilter {
    private finalClass<? > clazz;private final Set<String> includes = new HashSet<>();
    private final Set<String> excludes = new HashSet<>();

    public MyPropertyPreFilter(String... properties) {
        this((Class) null, properties);
    }

    public MyPropertyPreFilter(Class
        clazz, String... properties) {
        this.clazz = clazz;
        String[] var3 = properties;
        int var4 = properties.length;

        for (int var5 = 0; var5 < var4; ++var5) {
            String item = var3[var5];
            if(item ! =null) {
                this.includes.add(item); }}}public Set<String> getIncludes(a) {
        return includes;
    }
    public Set<String> getExcludes(a) {
        return excludes;
    }
    @Override
    public boolean apply(JSONSerializer jsonSerializer, Object source, String fieldName) {
        if (source == null) {
            return true;
        }
        if(clazz ! =null && !clazz.isInstance(source)) {
            return true;
        }

        SerialContext serialContext = jsonSerializer.getContext();
        String rankName = serialContext.toString();
        rankName = rankName + "." + fieldName;
        rankName = rankName.replace("$."."");// remove the leading $.
        rankName = rankName.replaceAll("\\[\\d+\\]"."");// get rid of [] if it is an array
        if (this.excludes.contains(rankName)) {
            return false;
        }

        if (includes.size() == 0 || includes.contains(rankName)) {
            return true;
        }

        return false; }}Copy the code

Basically, we changed the name of the judgment to include the hierarchical name

        MyPropertyPreFilter myPropertyPreFilter = new MyPropertyPreFilter();
        myPropertyPreFilter.getExcludes().add("orderAmount");
        myPropertyPreFilter.getExcludes().add("resource.producer");
        myPropertyPreFilter.getExcludes().add("contacts.linkPhone");
        String jsonString = JSON.toJSONString(order, myPropertyPreFilter);
        System.out.println(jsonString);
Copy the code

Results of the run:

{"contacts": [{"age":30."name":"burgxun"}, {"age":30."name":"And"}]."name":"This is the name of a special tag."."orderNO":"BU0000452"."resource": {"name":"Suzhou Two Flights to Japan"."type":1}}
Copy the code

The rest of the filters are introduced

  • PropertyPreFilter: Determines whether serialization is performed according to the PropertyName
  • PropertyFilter: Determines serialization based on PropertyName and PropertyValue
  • NameFilter: Changes the Key. If the Key needs to be changed, the value returned by process can be changed
  • ValueFilter: Modifies a Value
  • BeforeFilter: Add content first when serializing
  • AfterFilter: Add content at the end of serialization

I’m not using all of these filters here just PropertyPreFilter and ValueFilter;

The use of PropertyPreFilter scenarios I described above;

ValueFilter filter is where I use it, our data for storage performance consideration, will put some relatively large strings, into binary compressed storage Nosql database; At this point we can’t just use the value directly and we have to process it ourselves and render it in a Jons string. This is also very simple as long as the ValueFilter interface can be implemented!

conclusion

Nonsense so much ~ also do not know can bring you a little harvest!

In this project, I changed several versions of the project, optimized several versions of the final implementation is more general implementation is easier to understand ~

Finally by the way, if you encounter a problem, first go to see the official document. If you can’t find it, you should also find a better website to view it. Baidu really is a push and push of duplicate copy without changing ~ no value

Hopefully this article has given you some help with fastJSON filtering

The fastjson GiitHub wiki is located at github.com/alibaba/fas…