preface

To be a good Android developer, you need a complete set ofThe knowledge systemHere, let us grow together into what we think ~.

First, pre-knowledge

1. Functional programming

1) What is functional programming?

Object-oriented programming abstracts data, while functional programming abstracts behavior. In the real world, data and behavior coexist, and so do programs.

2) Why learn functional programming?

Data processing with functions (behaviors) is the cornerstone of learning big data.
Good efficiency (concurrent execution),
Accomplish a feature using less code.
The idea of turning objects into function-oriented programming is difficult and requires a lot of practice

2. Java 1.8 Lambda expressions

What is a Lambda expression?

Lambda is an anonymous function, that is, a function without a function name, which simplifies the use of anonymous delegates and makes code more concise.

2), two simple examples of Lambda expressions

// Anonymous inner class
Runnable r = new Runnable() {
    @Override
    public void run(a) {
        System.out.print("hello toroot"); }};//lambda
Runnable r2 = ()->System.out.print("hello toroot");

// Anonymous inner class
TreeSet<String> ts = new TreeSet<>(new Comparator<String>() {
    @Override
    public int compare(String o1, String o2) {
        returnLong.compare(o1.length(),o2.length()); }});//lambda
TreeSet<String> ts2 = new TreeSet<>((o1,o2)-> Long.compare(o1.length(),o2.length()));
Copy the code

3) Lambda expression syntax

Lambda expressions introduce a new syntactic element and operator into the Java language. The operator is “->”, which is called the Lambda or clipping operator.

It breaks Lambda into two parts:

Left: specifies all the arguments needed for a Lambda expression.
Right: specifies the body of the Lambda, which is the function that the Lambda expression is to perform.

4) Syntax format of Lambda expressions

1, no parameter, no return value:() -> System.out.println("Hello Lambda!" );
2, take one argument and return no value:(x) -> System.out.println(x)
3. If there is only one argument, the parentheses can be omitted:x -> System.out.println(x)
If the Lambda body has more than two arguments, it has a return value, and there are multiple statements in the Lambda body:

		Comparator<Integer> com = (x, y) -> {
			System.out.println("Functional interface");
			return Integer.compare(x, y);
		};
Copy the code

If there is only one statement in the Lambda body, return and braces can be omitted:Comparator<Integer> com = (x, y) -> Integer.compare(x, y);
The data type of the argument list of a Lambda expression can be omitted because the JVM compiler can infer the data type from the context, known as “type inference” :(Integer x, Integer y) -> Integer.compare(x, y);

5) Functional interface

Lambda expressions require support for “functional interfaces.” A functional interface is an interface that contains only one abstract method. You can use the @functionalinterface annotation, which checks for a FunctionalInterface. The following is an example of a functional interface:

@FunctionalInterface
public interface MyFun {
    public double getValue(a);
}

@FunctionalInterface
public interface MyFun<T> {
    public T getValue(T t);
}

public static void main(String[] args) {
    String newStr = toUpperString((str)->str.toUpperCase(),"toroot");
    System.out.println(newStr);
}

public static String toUpperString(MyFun<String> mf,String str) {
    return mf.getValue(str);
}
Copy the code

6) Java built-in functional interface

interface	parameter	The return type	The sample
Predicate	T	boolean	Is this problem right?
Consumer	T	void	Output a value
Function<T,R>	T	R	Get the name of the Person object
Supplier	None	T	The factory method
UnaryOperator	T	T	Logic is not (!)
BinaryOperator	(T, T)	T	Find the product of two numbers (*)

7) Method reference

Method references can be used when an operation to be passed to a Lambda body already has a method implemented. Method references use the “::” operator to separate the method name from the object or class name. It mainly has the following three uses:

Object :: instance method
Class :: static methods
Class :: instance methods

8) When will it be available: :Method references (emphasis)

When we use a Lambda expression, the part to the right of the “->” is the code to execute, that is, the function to accomplish. This part can be called the Lambda body. Sometimes, when we want to implement the abstract method of a functional interface, but there is already a class that implements the desired functionality, we can use method references to directly use the functionality of the existing class. The sample code looks like this:

Person p1 = new Person("Av".18.90);
        Person p2 = new Person("King".20.0);
        Person p3 = new Person("Lance".17.100);
        List<Person> list = new ArrayList<>();
        list.add(p1);
        list.add(p2);
        list.add(p3);

        // Here we need to compare the persons in the list by age
        // The most common way to do this is
        // sort(List<T> list, Comparator<? super T> c)
        Since the second argument to our sort method is an interface, we need to implement an anonymous inner class
        Collections.sort(list, new Comparator<Person>() {
            @Override
            public int compare(Person person1, Person person2) {
                returnperson1.getAge().compareTo(person2.getAge()); }});// The second argument is a @functionalInterface function, so we can use lambda
        Collections.sort(list, (person1,person2) -> p1.getScore().compareTo(p2.getAge()));
        // Since the second argument can be lambda,
        // But there is a Comparator.comparing code that does this
        // At this point we can use method reference
        /** * Important: * When we want to implement the abstract method of a functional interface, but there is already a class that implements the desired function, * we can use method references to implement the function directly using the existing class. * /
        Collections.sort(list, Comparator.comparing(Person::getAge));

        System.out.println(list);
Copy the code

Examples of other built-in Java functional interfaces are as follows:

public static void main(String[] args) {
    Consumer<String>  c = x->System.out.println(x);
    / / is equivalent to
    Consumer<String> c2 = System.out::print;
}

public static void main(String[] args) {
    BinaryOperator<Double> bo = (n1,n2) ->Math.pow(n1,n2);
    BinaryOperator<Double> bo2 = Math::pow;
}

public static void main(String[] args) {
    BiPredicate<String,String> bp = (str1,str2) ->str1.equals(str2);
    BiPredicate<String,String> bp2 = String::equals;
}
Copy the code

Note: Use ClassName::methodName when the first argument you need to reference a method is the calling object, and the second argument is the second argument (or no argument) that you need to reference the method.

9) Constructor references

Format: ClassName :: new combines with functional interfaces, automatically compatible with methods in functional interfaces. A constructor reference can be assigned to a defined method, but the constructor argument list is the same as the argument list for the abstract method in the interface. Here is an example:

public static void main(String[] args) {
    Supplier<Person> x = ()->new Person();
    Supplier<Person> x2 = Person::new;
}

public static void main(String[] args) {
    Function<String,Person> f  = x->new Person(x);
    Function<String,Person> f2 = Person::new;
}
Copy the code

10) Array references

Format: type[] :: new, as shown below:

public static void main(String[] args) {
   Function<Integer,Person[]> f  = x->new Person[x];
   Function<Integer,Person[]>  f2 = Person[]::new;
}
Copy the code

3, Stream API

1) what is Stream?

A Stream is a data channel used to manipulate sequences of elements generated by data sources (collections, arrays, and so on). Remember, “Collections are about data, streams are about computation!”

2) Features

1) Stream does not store elements by itself.
2) Stream does not change the source object. Instead, they return a new Stream holding the result.
3) The Stream operation is delayed. This means they wait until they need results.

3) a Stream instance

Take the names of all people older than 18, sort them by dictionary, and print them to the console as follows:

   private static  List<Person> persons = Arrays.asList(
            new Person("CJK".19."Female"),
            new Person("BODUO".20."Female"),
            new Person("JZ".21."Female"),
            new Person("anglebabby".18."Female"),
            new Person("huangxiaoming".5."Male"),
            new Person("ROY".18."Male"));public static void main(String[] args) throws IOException {
 persons.stream().filter(x-    	>x.getAge()>=18).map(Person::getName).sorted().forEach(System.out::println);
}
Copy the code

4) Three steps of Stream operation

Create a Stream: a data source (e.g., collection, array) and fetch a Stream.
2. Intermediate operation: an intermediate operation chain processes the data of the data source.
3. Termination operation (terminal operation) : A termination operation that performs an intermediate chain of operations and produces a result.

1. Create Steam

There are four main ways to create a stream, with sample code as follows:

@Test
public void test1(a){
    Collection provides two methods stream() and parallelStream().
    List<String> list = new ArrayList<>();
    Stream<String> stream = list.stream(); // Get a sequential stream
    Stream<String> parallelStream = list.parallelStream(); // Get a parallel stream

    //2. Get an array stream from stream() of Arrays
    Integer[] nums = new Integer[10];
    Stream<Integer> stream1 = Arrays.stream(nums);

    //3. Static method of() on Stream
    Stream<Integer> stream2 = Stream.of(1.2.3.4.5.6);
    
    //4. Create an infinite stream
    / / iteration
    Stream<Integer> stream3 = Stream.iterate(0, (x) -> x + 2).limit(10);
    stream3.forEach(System.out::println);

    / / generated
    Stream<Double> stream4 = Stream.generate(Math::random).limit(2);
    stream4.forEach(System.out::println);
}
Copy the code

2. Intermediate operation

1),Screening and sectioning
- Filter: Receives a Lambda, excluding certain elements from the stream.
- Limit: Truncates a stream so that it does not exceed a given number of elements.
- Skip (n) : Skip elements, returning a stream with the first n elements thrown away. If there are less than n elements in the stream, an empty stream is returned. Complementary to limit(n).
- Distinct: Filter to remove duplicate elements by hashCode() and equals() of the elements generated by the stream.
2),mapping
- Map: Receives lambdas, converts elements to other forms, or extracts information. Take as an argument a function that is applied to each element and maps it to a new element, similar to the Map syntax of Python and Go.
- FlatMap: Takes a function as an argument, replaces each value in the stream with another stream, and joins all streams into one stream.
3),The sorting
- Sorted () : sort naturally.
- Sorted (Comparator com) : Custom sort.

Here, we give some common usage examples, as follows:

Integer[] ary = {1,2,3,4,5,6,7,8,9,10}

List<Integer> collect = Arrays.stream(ary).skip(2).limit(3).collect(Collectors.toList());
Copy the code

2, there is a Integer array [] ary =,2,2,3,4,5,6,6,7,8,8,9,10 {1}, and get the even, and duplication.

List<Integer> list = Arrays.stream(ary).filter(x -> x % 2= =0).distinct().collect(Collectors.toList());
Set<Integer> integerSet = Arrays.stream(ary).filter(x -> x % 2= =0).collect(Collectors.toSet());
Copy the code

1,2,3,4,5… 1,2,3,4,5… 12)

Integer [] [] ary = {,8,4,7,5 {3}, {9,1,6,2}, {0,10,12,11}};

Arrays.stream(ary).flatMap(item->Arrays.stream(item)).sorted().forEach(System.out::println);
Copy the code

3) Terminate the operation

Terminal operations generate results from the pipeline of streams. The result can be any value that is not a stream, such as List, Integer, or even void.

1. Find and match

interface	instructions
allMatch(Predicate p)	Check that all elements match
anyMatch(Predicate p)	Check that at least one element matches
noneMatch(Predicate p)	Check that all elements are not matched
findFirst()	Returns the first element
findAny()	Returns any element in the current stream
count()	Returns the total number of elements in the stream
max(Comparator c)	Returns the maximum value in the stream
min(Comparator c)	Returns the minimum value in the stream
forEach(Consumer c)	The iteration

2, reduction

Reduce (T iden, BinaryOperator b) can combine elements ina stream repeatedly to obtain a value. Returns the Optional. For example, an example code using reduce to calculate the total score of all staff students would look like this:

   Integer all = persons.stream().map(Person::getScore).reduce((integer, integer2) -> integer + integer2).get()
Copy the code

3, collect

Collect (Collector C) Converts streams to other forms. It receives an implementation of the Collector interface, a method for summarizing elements in the Stream.
The implementation of methods in the Collector interface determines how collection operations (such as collecting lists, sets, maps) are performed on streams.
The Collectors class provides many static methods that make it easy to create common collector instances.

The relevant Stream API to collect and its instance code are shown below:

1) toList List collects elements from the stream into List:

List<Person> emps= list.stream().collect(Collectors.toList());
Copy the code

ToSet Set collects elements from the stream into a Set:

Set<Person> emps= list.stream().collect(Collectors.toSet());
Copy the code

ToCollection Collection collects elements from the stream into the created Collection:

Collection<Person> emps=list.stream().collect(Collectors.toCollection(ArrayList::new));
Copy the code

4) Counting Long Count the number of elements in the stream:

long count = list.stream().collect(Collectors.counting());
Copy the code

5) summing Int Integer Specifies the Integer attributes of the elements in the stream.

int total=list.stream().collect(Collectors.summingInt(Person::getAge));
Copy the code

We would have calculated the average value of the Integer attribute of the element in the stream:

double avg= list.stream().collect(Collectors.averagingInt(Person::getAge));
Copy the code

SummarizingInt IntSummaryStatistics Collects statistics about the Integer attribute in the summarizingStream. Such as average value:

Int SummaryStatisticsiss= list.stream().collect(Collectors.summarizingInt(Person::getAge));
Copy the code

8) Joining String:

String str= list.stream().map(Person::getName).collect(Collectors.joining());
Copy the code

9), maxBy Optional Select maximum value from comparator:

Optional<Person> max= list.stream().collect(Collectors.maxBy(comparingInt(Person::getSalary)));
Copy the code

10), minBy Optional select minimum value from comparator:

Optional<Person> min = list.stream().collect(Collectors.minBy(comparingInt(Person::getSalary)));
Copy the code

Reducing a type to a single value, starting with an initial value as an accumulator and combining it with elements in the stream one by one using a BinaryOperator:

int total=list.stream().collect(Collectors.reducing(0, Person::getSalary, Integer::sum));
Copy the code

CollectingAndThen converts the type returned by the function to wrap another collector around its result

int how= list.stream().collect(Collectors.collectingAndThen(Collectors.toList(), List::size));
Copy the code

13), groupingBy Map

according to some attribute value, attribute is K, result is V:
,>

Map<Person.Status, List<Person>> map= list.stream().collect(Collectors.groupingBy(Person::getStatus));
Copy the code

14), partitioningBy Map
,>

Map<Boolean,List<Person>>vd= list.stream().collect(Collectors.partitioningBy(Person::getManage));
Copy the code

4. Termination of operation practice cases

1) Select all the names of the Person object and place them in the List:

List<String> collect2 = persons.stream().map(Person::getName).collect(Collectors.toList());
Copy the code

2, find the average score, total score, highest score, lowest score, number of scores of Person object set:

IntSummaryStatistics collect = persons.stream().collect(Collectors.summarizingInt(Person::getScore));
System.out.println(collect);
Copy the code

3, according to the results of the grouping, pass to put one group, failed to put another group:

Map<Boolean, List<Person>> collect1 = persons.stream().collect(Collectors.partitioningBy(person -> person.getScore() >= 60));
System.out.println(collect1);
Copy the code

4, count the number of words in aA. TXT:

public static void main(String[] args) throws IOException {
    InputStream resourceAsStream = Person.class.getClassLoader().getResourceAsStream("aa.txt");
    BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(resourceAsStream));
    bufferedReader.lines().flatMap(x->Stream.of(x.split(""))).sorted().collect(Collectors.groupingBy(String::toString)).forEach((a,b)-> System.out.println(a+":"+b.size()));
    bufferedReader.close();
}
Copy the code

4. Complex generics

1) What are generics?

Generics, or parameterized types. That is, a type is parameterized from an original concrete type, similar to a variable parameter in a method, in which the type is also defined as a parameter (called a type parameter), and then passed in the concrete type (type argument) when used or called.

2) Benefits of generics

Suitable for multiple data types to execute the same code.
Types in generics are specified at usage time and do not require casts.

3) Generic classes and interfaces, generic methods

The nature of generics is to parameterize types (to control the types of parameters that are specifically restricted by the different types specified by generics without creating new types). These parameter types can be used in classes, interfaces, and methods, called generic classes, generic interfaces, and generic methods, respectively.

A generic class

Introduce a type variable T (any other uppercase letter will do, but the usual ones are T, E, K, V, etc.), enclose it in <>, and place it after the class name. Generic classes are allowed to have multiple type variables. Common sample code looks like this:

public class NormalGeneric<K> {
    private K data;

    public NormalGeneric(a) {}public NormalGeneric(K data) {
        this.data = data;
    }

    public K getData(a) {
        return data;
    }

    public void setData(K data) {
        this.data = data; }}Copy the code

public class NormalGeneric2<T.K> {
    private T data;
    private K result;

    public NormalGeneric2(a) {}public NormalGeneric2(T data) {
        this(a);this.data = data;
    }
    
    public NormalGeneric2(T data, K result) {
        this.data = data;
        this.result = result;
    }

    public T getData(a) {
        return data;
    }

    public void setData(T data) {
        this.data = data;
    }

    public K getResult(a) {
        return result;
    }

    public void setResult(K result) {
        this.result = result; }}Copy the code

A generic interface

Generic interfaces are basically the same as generic classes. The sample code looks like this:

public interface Genertor<T> {
    public T next(a);
}
Copy the code

However, classes that implement generic interfaces can be implemented in two ways:

1. No generic argument is passed

When you create an instance of a class, you need to specify a specific type:

public class ImplGenertor<T> implements Genertor<T> {
    @Override
    public T next(a) {
        return null; }}Copy the code

2. Pass in the generic argument

When you create an instance of a class, it’s the same as a normal class.

public class ImplGenertor2 implements Genertor<String> {
    @Override
    public String next(a) {
        return null; }}Copy the code

Generic method

Generic methods are defined between modifiers and return values. The sample code looks like this:

public <T> T genericMethod(T... a){
    return a[a.length/2];
}
Copy the code

Generic methods indicate the specific type of the generic when calling the method. Generic methods can be used anywhere and in any scenario, including ordinary classes and generic classes.

The difference between generic and generic methods defined in a generic class

In the ordinary method:

// Although generics are used in the method, it is not a generic method.
// This is just a normal member method of the class, but its return value is the declaration of the generic class already declared the generic type.
// So we can continue to use the T generic in this method.
public T getKey(a){
    return key;
}
Copy the code

In generic methods:

/** * This is a true generic method. * The first 
      
        between public and the return value is essential, indicating that this is a generic method and declaring a generic T * this T can appear anywhere in the generic method and the number of generics can be as many as you want. * /
      
public <T,K> K showKeyName(Generic<T> container){
    // ...
}
Copy the code

4) Qualify type variables

public class ClassBorder<T extends Comparable> {... }public class GenericRaw<T extends ArrayList&Comparable> {... }Copy the code

–

: T indicates the subtype of the type to be bound. Comparable indicates the binding type. The subtypes and binding types can be classes or interfaces.

Extends extends Comparable&Serializable allows both sides of extends, such as T,V extends Comparable&Serializable.
Note that only one class is allowed in a qualified type, and if there is one, it must be the first class in the qualified list.
Qualified type variables can be used on generic methods or generic classes.

5) Constraints and limitations in generics

Type parameters cannot be instantiated with primitive types.
2. Runtime type queries apply only to primitive types.
3. Invalidation of type variables in the static context of generic classes: Type variables cannot be referenced in static fields or methods. Because generics are not known until the object is created, object creation code executes the static part first, constructors second, and so on. So the static part is executed before the object is initialized. If you refer to generics in the static part, the virtual machine will no doubt know what it is because the class has not been initialized yet.
You cannot create arrays of parameterized types, but you can define arrays of parameterized types.
Type variables cannot be instantiated.
6. You cannot use try-catch to catch instances of generic classes.

6) Inheritance rules for generic types

Generic classes can inherit or extend other generic classes, such as List and ArrayList:

private static class ExtendPair<T> extends Pair<T>{... }Copy the code

7) Wildcard type

? extends X:Represents an upper bound on a type whose argument is a subclass of X.
? super X:Represents the lower bound of the type, and the type argument is the superclass of X.

? extends X

If you provide methods for get and set parameters, the set method is not allowed to be called and will cause compilation errors, while the GET method is fine.

? Extends X is an upper bound on a type. The type argument is a subclass of X, so it’s safe to say that the get method returns an X (whether X or a subclass of X) that the compiler knows for sure. But the set method only knows that it is passing in an X, not a subclass of X.

As a result,? Extends X is primarily used to securely access data. X and its subtypes can be accessed and non-NULL data cannot be written.

? super X

The set method can be called if a get or set parameter variable is provided, and only X or a subclass of X can be passed in. The get method only returns a value of type Object.

? Super X is the lower bound of the type, and the type argument is X’s superclass (including X itself), so it is safe to say that get must return X’s superclass. No, but it’s safe to say that Object must be its superclass, so the get method returns Object. The compiler knows for sure. For the set method, the compiler does not know the exact type it needs, but X and subclasses of X can be safely converted to X.

As a result,? Super X is primarily used for securely writing data, and X and its subtypes can be written.

Unqualified wildcards?

There is no restriction on the type. As a parent of all types, such as ArrayList
.

8) How do virtual machines implement generics?

The idea of generics began to take root in the Template of C++ language. When there was no version of generics in Java language, type generalization could only be realized through the combination of Object being the parent class of all types and type casting.

Since all types in the Java language inherit from java.lang.Object, it is possible to turn an Object into any Object. But because of the infinite possibilities, only the programmer and the running virtual machine know what type of Object this Object is. At compile time, there is no way for the compiler to check whether an Object was successfully cast, and if the programmer is left to ensure that this operation is correct, much of the risk of ClassCastException is transferred to the program runtime.

In addition, generics appear to be used in C#/C++ and Java in the same way, but the implementation is fundamentally different. Generics in C# are in the program source code and in the compiled IL (Intermediate Language, where generics are placeholders). List < int > and List < String > are two different types that are generated at runtime and have their own virtual method tables and type data. This implementation is called type inflation. Generics implemented based on this method are called true generics.

Generics in The Java language, on the other hand, exist only in the program source code. In the compiled bytecode file, they are replaced with the original Raw Type (also known as Raw Type), and forced transformation code is inserted in the corresponding place. Therefore, for the Runtime Java language, ArrayList < int > is the same class as ArrayList < String >, so generics technology is actually a syntactic sugar of the Java language. The implementation method of generics in the Java language is called type erasing, and generics based on this method are called pseudo-generics. When you compile a piece of Java code into a Class file and then decompile it using a bytecode decompiler, you will find that the generics are gone and the program is written as it was before Java generics, and the generic types are returned to their native types.

With the introduction of Java generics, method calls under various scenarios (virtual machine parsing, reflection, and so on) can have an impact on the old infrastructure and new requirements, such as how to retrieve incoming parameterized types in generic classes. As a result, the JCP organization modified the virtual machine specification to introduce new attributes such as Signature and LocalVariableTypeTable to solve the problem of identifying parameter types that come with generics. Signature is the most important of these attributes. It stores the bytecode signature of a method. The parameter type stored in this property is not a primitive type, but contains the information of the parameterized type. The modified VM specifications require that all VMS that can recognize Class files of version 49.0 or higher correctly recognize the Signature parameter.

Finally, we can conclude from the appearance of the Signature attribute that the so-called erasure is only the bytecode in the Code attribute of the method. In fact, the metadata still retains the generic information, which is the basic basis of the parameterized type through reflection.

First acquaintance with ByteX

ByteX uses pure Java to write the source code, it is a bytecode pegs platform based on Gradle Transform API and ASM.

Gradle Clean :example: Assemblerelease-dorg.gradle. debug=true –no-daemon

1, the advantages of

1) Automatic integration into other hosts and integration of plug-ins into a single MainTransformFlow, combined with multi-threaded concurrent processing of class files, avoiding linear growth of additional packaging time.
2) Fully decoupled between plug-in and host, facilitating collaborative development.
3) Common Module provides universal code reuse, and each plug-in only needs to focus on its own bytecode staking logic.

2. MainTransformFlow basic process

The normal process of MainTransformFlow implements MainProcessHandler, which iterates through all classes in two engineering builds.

1) For the first time, traverse traverse and traverseAndroidJar to form the complete class diagram.
2) For the second time, execute transform: iterate through all the build products in the project again, and process the class file and output it.

3. How to customize independent TransformFlow?

Rewrite the IPlugin’s provideTransformFlow.

4. Class diagram objects

Context.getclassgraph () gets the class graph object. The two TransformFlow class diagrams are isolated.

5, MainProcessHandler

Process it by overwriting the process method and registering your own FlieProcessor.
FileProcessor uses the chain of responsibility model, where each class file is processed through a series of FileProcessors.

6, IPlugin. HookTransformName ()

Three, ByteX plug-in platform construction process exploration

Bytex works in Gradle’s build process after adding the Apply plugin: ‘bytex’. The plugin id here is bytex. We go to the bytex.properties file and look at the mapped implementation class, as shown below:

implementation-class=com.ss.android.ugc.bytex.base.ByteXPlugin
Copy the code

As you can see, the implementation class of Bytex is ByteXPlugin, and its source code is as follows:

public class ByteXPlugin implements Plugin<Project> {
    @Override
    public void apply(@NotNull Project project) {
        / / 1
        AppExtension android = project.getExtensions().getByType(AppExtension.class);
        / / 2
        ByteXExtension extension = project.getExtensions().create("ByteX", ByteXExtension.class);
        / / 3
        android.registerTransform(new ByteXTransform(newContext(project, android, extension))); }}Copy the code

First, at note 1, get the AppExtension instance that Android provides for the App. Then, in comment 2, get an instance of the extension property ByteXExtension created by ByteX itself. Finally, in comment 3, register the ByteXTransform instance. ByteXTransform inherits the abstract class CommonTransform, which implements the key transform method.

@Override
public final void transform(TransformInvocation transformInvocation) throws TransformException, InterruptedException, IOException {
    super.transform(transformInvocation);
    // 1. If the mode is not incremental, the output directory is clear.
    if(! transformInvocation.isIncremental()) { transformInvocation.getOutputProvider().deleteAll(); }// 2. Obtain the transformContext instance.
    TransformContext transformContext = getTransformContext(transformInvocation);
    // 3, initialize HtmlReporter (generate ByteX build generated log HTML file)
    init(transformContext);
    // Filter plugins without plugins enabled.
    List<IPlugin> plugins = getPlugins().stream().filter(p -> p.enable(transformContext)).collect(Collectors.toList());
    Timer timer = new Timer();
    // create an instance transformEngine.
    TransformEngine transformEngine = new TransformEngine(transformContext);
    try {
        if(! plugins.isEmpty()) {// 6. Prioritize each TransformFlow using PriorityQueue (the corresponding implementation class MainTransformFlow is added here).
            Queue<TransformFlow> flowSet = new PriorityQueue<>((o1, o2) -> o2.getPriority() - o1.getPriority());
            MainTransformFlow commonFlow = new MainTransformFlow(transformEngine);
              flowSet.add(commonFlow);
            for (int i = 0; i < plugins.size(); i++) {
                Register the MainTransformFlow for each Plugin by adding the Handlers list to the MainProcessHandler of each Plugin.
                IPlugin plugin = plugins.get(i);
                TransformFlow flow = plugin.registerTransformFlow(commonFlow, transformContext);
                if (!flowSet.contains(flow)) {
                    flowSet.add(flow);
                }
            }
            while(! flowSet.isEmpty()) { TransformFlow flow = flowSet.poll();if(flow ! =null) {
                    if (flowSet.size() == 0) {
                        flow.asTail();
                    }
                    Executes each TransformFlow run method with the specified priority. By default, there is only one MainTransformFlow instance.
                    flow.run();
                    // 9. Get the graph object in the stream and clear it.
                    Graph graph = flow.getClassGraph();
                    if(graph ! =null) {
                        // Clear the class diagram. We won't use it anymoregraph.clear(); }}}}else {
            transformEngine.skip();
        }
        / / 10
        afterTransform(transformInvocation);
    } catch (Throwable throwable) {
        LevelLog.sDefaultLogger.e(throwable.getClass().getName(), throwable);
        throw throwable;
    } finally {
        for (IPlugin plugin : plugins) {
            try {
                plugin.afterExecute();
            } catch (Throwable throwable) {
                LevelLog.sDefaultLogger.e("do afterExecute", throwable);
            }
        }
        transformContext.release();
        release();
        timer.record("Total cost time = [%s ms]");
        if(BooleanProperty.ENABLE_HTML_LOG.value()) { HtmlReporter.getInstance().createHtmlReporter(getName()); HtmlReporter.getInstance().reset(); }}}Copy the code

In notes 7, call the plugin. RegisterTransformFlow method, its source code is as follows:

@Nonnull
@Override
public final TransformFlow registerTransformFlow(@Nonnull MainTransformFlow mainFlow, @Nonnull TransformContext transformContext) {
    if (transformFlow == null) {
        transformFlow = provideTransformFlow(mainFlow, transformContext);
        if (transformFlow == null) {
            throw new RuntimeException("TransformFlow can not be null."); }}return transformFlow;
}
Copy the code

Here we continue to call the provideTransformFlow method, whose source code looks like this:

/**
 * create a new transformFlow or just return mainFlow and append a handler.
 * It will be called by {@link IPlugin#registerTransformFlow(MainTransformFlow, TransformContext)} when
 * handle start.
 *
 * @param mainFlow         main TransformFlow
 * @param transformContext handle context
 * @return return a new TransformFlow object if you want make a new flow for current plugin
 */
protected TransformFlow provideTransformFlow(@Nonnull MainTransformFlow mainFlow, @Nonnull TransformContext transformContext) {
    return mainFlow.appendHandler(this);
}
Copy the code

As you can see, the MainProcessHandler of each Plugin is added to the Handlers list in MainTransformFlow by calling the MainFlow.appendHandler (this) method.

In comment 8, each TransformFlow run method is executed with the specified priority, and there is only one MainTransformFlow instance by default. We see the Run method of MianTransformFlow:

@Override
public void run(a) throws IOException, InterruptedException {
    try {
        / / 1
        beginRun();
        / / 2
        runTransform();
    } finally {
        / / 3endRun(); }}Copy the code

First, in comment 1, the beginRun method is called, which is implemented as follows:

// AbsTransformFlow
protected void beginRun(a) {
    transformEngine.beginRun();
}

// TransformEngine
public void beginRun(a){
    context.markRunningState(false);
}

// TransformContext
private final AtomicBoolean running = new AtomicBoolean(false);

void markRunningState(boolean running) {
    this.running.set(running);
}
Copy the code

Finally, an AtomicBoolean instance is used in the TransformContext instance to indicate whether MainTransformFlow is running.

The runTransform method is then executed in comment 2, and this is where the transform is actually executed. The source code looks like this:

private void runTransform(a) throws IOException, InterruptedException {
    if (handlers.isEmpty()) return;
    Timer timer = new Timer();
    timer.startRecord("PRE_PROCESS");
    timer.startRecord("INIT");
    Initialize each handler in the Handlers list.
    for (MainProcessHandler handler : handlers) {
        handler.init(transformEngine);
    }
    timer.stopRecord("INIT"."Process init cost time = [%s ms]");
    Traverse procedures are only executed if traverse is not skipped and only the Transform method is executed.
    if(! isOnePassEnough()) {if(! handlers.isEmpty() && context.isIncremental()) { timer.startRecord("TRAVERSE_INCREMENTAL");
            // if it is incremental, traverseArtifactOnly calls the traverseIncremental method of the MainProcessHandler corresponding to each plugin. This is where the classFileAnalyzer.handle method is eventually called to traverse the distribution.
            traverseArtifactOnly(getProcessors(Process.TRAVERSE_INCREMENTAL, new ClassFileAnalyzer(context, Process.TRAVERSE_INCREMENTAL, null, handlers)));
            timer.stopRecord("TRAVERSE_INCREMENTAL"."Process project all .class files cost time = [%s ms]");
        }
        handlers.forEach(plugin -> plugin.beforeTraverse(transformEngine));
        timer.startRecord("LOADCACHE");
        // create a CachedGraphBuilder object: a class graph builder object that can cache the class graph.
        GraphBuilder graphBuilder = new CachedGraphBuilder(context.getGraphCache(), context.isIncremental(), context.shouldSaveCache());
        if(context.isIncremental() && ! graphBuilder.isCacheValid()) {// If it is an incremental update && graphBuilder's cache is invalid, request a non-incremental run directly.
            context.requestNotIncremental();
        }
        timer.stopRecord("LOADCACHE"."Process loading cache cost time = [%s ms]");
        Running. set(true) is called internally to indicate the running status.
        running();
        if(! handlers.isEmpty()) { timer.startRecord("PROJECT_CLASS");
            // Execute traverseArtifactOnly to call the MainProcessHandler traverse method of each plugin. This is where the classFileAnalyzer.handle method is eventually called to traverse the distribution.
            traverseArtifactOnly(getProcessors(Process.TRAVERSE, new ClassFileAnalyzer(context, Process.TRAVERSE, graphBuilder, handlers)));
            timer.stopRecord("PROJECT_CLASS"."Process project all .class files cost time = [%s ms]");
        }
        if(! handlers.isEmpty()) { timer.startRecord("ANDROID");
            // 7. Just traverse android.jar
            traverseAndroidJarOnly(getProcessors(Process.TRAVERSE_ANDROID, new ClassFileAnalyzer(context, Process.TRAVERSE_ANDROID, graphBuilder, handlers)));
            timer.stopRecord("ANDROID"."Process android jar cost time = [%s ms]");
        }
        timer.startRecord("SAVECACHE");
        // Create an mClassGraph instance.
        mClassGraph = graphBuilder.build();
        timer.stopRecord("SAVECACHE"."Process saving cache cost time = [%s ms]");
    }
    timer.stopRecord("PRE_PROCESS"."Collect info cost time = [%s ms]");
    if(! handlers.isEmpty()) { timer.startRecord("PROCESS");
        // execute each plugin's transform method.
        transform(getProcessors(Process.TRANSFORM, new ClassFileTransformer(handlers, needPreVerify(), needVerify())));
        timer.stopRecord("PROCESS"."Transform cost time = [%s ms]"); }}Copy the code

First, in comment 1, the loop calls each MainProcessHandler’s init method, which is used to initialize the implementation before the Transform begins.

The MainProcessHandler interface init method is a default method, which directly calls the init method of each pluign implementation (if plugin is not implemented, Just call the init method implemented by CommonPlugin: this is usually used to add unwanted files to the mWhiteList list), and some plugin preparation can be done here.

1. Just iterate over the product

traverseArtifactOnly(getProcessors(Process.TRAVERSE, new ClassFileAnalyzer(context, Process.TRAVERSE, graphBuilder, handlers)));
Copy the code

The source code for the getProcessors method is as follows:

private FileProcessor[] getProcessors(Process process, FileHandler fileHandler) {
    List<FileProcessor> processors = handlers.stream()
            .flatMap((Function<MainProcessHandler, Stream<FileProcessor>>) handler -> handler.process(process).stream())
            .collect(Collectors.toList());
    switch (process) {
        case TRAVERSE_INCREMENTAL:
            processors.add(0.newFilterFileProcessor(fileData -> fileData.getStatus() ! = Status.NOTCHANGED)); processors.add(new IncrementalFileProcessor(handlers, ClassFileProcessor.newInstance(fileHandler)));
            break;
        case TRAVERSE:
        case TRAVERSE_ANDROID:
        case TRANSFORM:
            processors.add(ClassFileProcessor.newInstance(fileHandler));
            processors.add(0.newFilterFileProcessor(fileData -> fileData.getStatus() ! = Status.NOTCHANGED && fileData.getStatus() ! = Status.REMOVED))break;
        default:
            throw new RuntimeException("Unknow Process:" + process);
    }
    return processors.toArray(new FileProcessor[0]);
}
Copy the code

The addition of processors is defined by deltas. The processing criteria are as follows:

TRAVERSE_INCREMENTAL:
- FilterFileProcessor:Follow a different procedure to filter out unwanted FileData.
- IncrementalFileProcessor:Used for incremental file processing.
TRAVERSE/TRAVERSE_ANDROID/TRANSFORM:
- FilterFileProcessor:Follow a different procedure to filter out unwanted FileData.
- ClassFileProcessor:Used to process.class files.

2. Just iterate through the Android Jar package

traverseAndroidJarOnly(getProcessors(Process.TRAVERSE_ANDROID, new ClassFileAnalyzer(context, Process.TRAVERSE_ANDROID, graphBuilder, handlers)));
Copy the code

3. Build the mClassGraph object

mClassGraph = graphBuilder.build();
Copy the code

4. Execute Transform

transform(getProcessors(Process.TRANSFORM, new ClassFileTransformer(handlers, needPreVerify(), needVerify())));
Copy the code

The source code for transform looks like this:

Protected AbsTransformFlow transform(FileProcessor... processors) throws IOException, InterruptedException { beforeTransform(transformEngine); transformEngine.transform(isLast, processors); afterTransform(transformEngine); return this; }Copy the code

1), beforeTransform (transformEngine)

// MainTransformFlow
@Override
protected AbsTransformFlow beforeTransform(TransformEngine transformEngine) {
    / / 1
    handlers.forEach(plugin -> plugin.beforeTransform(transformEngine));
    return this;
}
Copy the code

In comment 1, the beforeTransform method of each plugin is iterated to do some pre-transform work of its own.

2), transformEngine. Transform (isLast, processors)

// TranformEngine
public void transform(boolean isLast, FileProcessor... processors) {
    Schedulers.FORKJOINPOOL().invoke(new PerformTransformTask(context.allFiles(), getProcessorList(processors), isLast, context));
}
Copy the code

The source code for the shedulers.forkJoinpool () method is as follows:

public class Schedulers {
    private static final int cpuCount = Runtime.getRuntime().availableProcessors();
    private final static ExecutorService IO = new ThreadPoolExecutor(0, cpuCount * 3.30L, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>());

    / / 1
    private static final ExecutorService COMPUTATION = Executors.newWorkStealingPool(cpuCount);

    public static Worker IO(a) {
        return new Worker(IO);
    }

    public static Worker COMPUTATION(a) {
        return new Worker(COMPUTATION);
    }

    public static ForkJoinPool FORKJOINPOOL(a) {
        return(ForkJoinPool) COMPUTATION; }}Copy the code

As you can see, the final is to perform Executors. NewWorkStealingPool (cpuCount) method to generate a ForkJoinPool instance.

ForkJoinPool is parallel to ThreadPoolExecutor. ForkJoinPool thread pools are created to implement the idea of “divide and conquer” by breaking large tasks into smaller ones and then aggregating the results of the smaller tasks. The idea is very similar to MapReduce. In addition to divide-and-conquer, ForkJoinPool uses a work-stealing algorithm, in which all threads attempt to find and execute submitted tasks or subtasks created from other tasks. With this we can try to avoid the situation where a thread is “idle” after completing its task.

This then calls back the compute method of the PerformTransformTask instance.

@Override
protected void compute(a) {
    if (outputFile) {
        // 1. If it is the last TransformFlow, all FileTransformTasks are recursively called.
        List<FileTransformTask> tasks = source.map(cache -> new FileTransformTask(context, cache, processors)).collect(Collectors.toList());
        // 2. If the number of threads in the Pool is fixed, Fork (A) is assigned to B, and B is assigned to B to complete the task assigned by A. So the pattern above is equivalent to wasting a thread. Then, using invokeAll means that A assigns work to B, and BOTH A and B finish the work. This makes better use of thread pools and reduces execution time.
        invokeAll(tasks);
    } else {
        // call FileTransformTask recursively
        PerformTraverseTask traverseTask = newPerformTraverseTask(source, processors); invokeAll(traverseTask); }}Copy the code

In comment 1, all FileTransformTasks are called if it was the last TransformFlow. Note 2: For Fork/Join mode, if the number of threads in the Pool is fixed, then calling the sub-task Fork method is equivalent to A first assigning the task to B, then A does not work as the supervisor, AND B completes the task assigned by A. So the pattern above is equivalent to wasting a thread. Then, using invokeAll means that A assigns work to B, and BOTH A and B finish the work. This makes better use of thread pools and reduces execution time. The ForkJoinTask invokeAll method is invoked in comment 3, where the compute method is called:

@Override
protected void compute(a) {
    List<FileTraverseTask> tasks = source.map(cache -> new FileTraverseTask(cache, processors)).collect(Collectors.toList());
    / / 1
    invokeAll(tasks);
}
Copy the code

Continue to call back the compute method for all instances of FileTraverseTask at comment 1:

@Override
protected void compute(a) {
    List<TraverseTask> tasks = fileCache.stream().map(file -> new TraverseTask(fileCache, file, processors))
            .toList().blockingGet();
    / / 1
    invokeAll(tasks);
}
Copy the code

Continue to call back the compute method for all instances of TraverseTask at comment 1:

@Override
protected void compute(a) {
    try {
        Input input = new Input(fileCache.getContent(), file);
        ProcessorChain chain = new ProcessorChain(processors, input, 0);
        // proceed to ProcessorChain.
        chain.proceed(input);
    } catch (IOException e) {
        throw newRuntimeException(e); }}Copy the code

At comment 1, the proceed method of ProcessorChain is called. The source code is as follows:

@Override
public Output proceed(Input input) throws IOException {
    if (index >= processors.size()) throw new AssertionError();
    / / 1
    FileProcessor next = processors.get(index);
    
    return next.process(new ProcessorChain(processors, input, index + 1));
}
Copy the code

In comment 1, we get the first processor, FilterFileProcessor, from the list of processors, and call its process method. The source code is as follows:

@Override
public Output process(Chain chain) throws IOException {
    Input input = chain.input();
    if (predicate.test(input.getFileData())) {
        / / 1
        return chain.proceed(input);
    } else {
        return newOutput(input.getFileData()); }}Copy the code

Proceed (); proceed (ClassFileProcessor); proceed (ClassFileProcessor);

@Override
public Output process(Chain chain) throws IOException {
    Input input = chain.input();
    FileData fileData = input.getFileData();
    if (fileData.getRelativePath().endsWith(".class")) {
        // If fileData is a.class file, call the Handle method of ClassFileTransformer to process it.
        handler.handle(fileData);
    }
    / / 2,
    return chain.proceed(input);
}
Copy the code

In comment 1, if fileData is a.class file, the handle method of ClassFileTransformer is called to process it. The source code is as follows:

@Override
public void handle(FileData fileData) {
    try {
        byte[] raw = fileData.getBytes();
        String relativePath = fileData.getRelativePath();
        int cwFlags = 0;  //compute nothing
        int crFlags = 0;
        for (MainProcessHandler handler : handlers) {
            // 1. Set the default value of ClassWrite flag to ClassWriter.COMPUTE_MAXS.
            cwFlags |= handler.flagForClassWriter();
            if ((handler.flagForClassReader(Process.TRANSFORM) & ClassReader.EXPAND_FRAMES) == ClassReader.EXPAND_FRAMES) {
                crFlags |= ClassReader.EXPAND_FRAMES;
            }
        }
        ClassReader cr = new ClassReader(raw);
        ClassWriter cw = new ClassWriter(cwFlags);
        ClassVisitorChain chain = getClassVisitorChain(relativePath);
        if (needPreVerify) {
            // If prevalidation is required, set the responsibility list header and tail to AsmVerifyClassVisitor instance.
            chain.connect(new AsmVerifyClassVisitor());
        }
        if(handlers ! =null && !handlers.isEmpty()) {
            for (MainProcessHandler handler : handlers) {
                // 3. Execute all plugin's transforms. It will use chain.connect(new ReferCheckClassVisitor(Context)) internally
                if(! handler.transform(relativePath, chain)) { fileData.delete();return; }}}// 4, compatible with ClassNode processing mode
        ClassNode cn = new SafeClassNode();
        chain.append(cn);
        chain.accept(cr, crFlags);
        for (MainProcessHandler handler : handlers) {
            if(! handler.transform(relativePath, cn)) { fileData.delete();return;
            }
        }
        cn.accept(cw);
        if(! GlobalWhiteListManager.INSTANCE.shouldIgnore(fileData.getRelativePath())) { raw = cw.toByteArray();if (needVerify) {
                ClassNode verifyNode = new ClassNode();
                new ClassReader(raw).accept(verifyNode, crFlags);
                AsmVerifier.verify(verifyNode);
            }
            If the file is not whitelisted, put the ClassWriter data into fileData.fileData.setBytes(raw); }}catch (ByteXException e) {
        throw e;
    } catch (Exception e) {
        LevelLog.sDefaultLogger.e(String.format("Failed to handle class %s", fileData.getRelativePath()), e);
        if(! GlobalWhiteListManager.INSTANCE.shouldIgnore(fileData.getRelativePath())) {throwe; }}}Copy the code

At comment 2 to the process method of the ClassFileProcessor, the ProcessorChain proceed method is called, which calls back the Process method of the BackupFileProcessor instance, The source code is as follows:

@Override
public Output process(Chain chain) throws IOException {
    Input input = chain.input();
    // Just return the processed output file
    return new Output(input.getFileData());
}
Copy the code

In this pattern, all tasks of the PerformTraverseTask instance are iterated.

The afterTransform method of the MainTransformFlow instance is called.

@Override
protected AbsTransformFlow afterTransform(TransformEngine transformEngine) {
    handlers.forEach(plugin -> plugin.afterTransform(transformEngine));
    return this;
}
Copy the code

All of plugin’s afterTransform methods are iterated through here.

We then go back to CommonTransform’s transform method and, after executing MainTransformFlow’s run method, call the code in comment 9 to retrieve and clean the Graph-like graph object in the stream. Finally, the afterTransform method at comment 10 is executed to do the finishing work after the transform.

Four,

In this article, we explored the process of building the ByteX plug-in platform. From the source code implementation of ByteX, we can see the author’s flexible use of functional programming, Java 1.8 Lambda expressions, Java 1.8 Stream API, complex generics and other technologies, so that almost all seemingly 🐂 wheels, Its essence is to rely on a deep grasp of the basic technology. So how do you get to the level of deep mastery of basic technology? Only practice and review regularly.

Reference links:

1, ByteX
2. Use and precautions of Fork/Join framework with high concurrency
3. Understanding the JVM
4. Ideas for Java Programming

ByteX is the framework of Gradle plug-in.