background

There is a function in which several different RPC requests need to be called. At the beginning, we didn’t think anything of it, so all THE RPC requests were executed in serial. Later, it was found that some RPCS took a long time to return, so the interface of this function took a long time. The new JDK8 feature CompletableFuture is used to execute these different RPC requests asynchronously and return the request result after all RPC requests are completed.

When using a CompletableFuture, there is no custom thread pool. The default is ForkJoinPool. Let’s look at the pseudocode:

CompletableFuture task1 = CompletableFuture. RunAsync (() - > {/ * * * will call an RPC request here, and the mechanism of RPC in the process of the request processing will pass SPL load to specify the implementation of the interface, The jar where this interface resides resides in web-info /lib */ system.out.println (" Task 1 execute "); }); CompletableFuture task2 = CompletableFuture. RunAsync (() - > {System. Out. Println (" 2 task execution "); }); CompletableFuture task3 = CompletableFuture. RunAsync (() - > {System. Out. Println (" 3 task execution "); }); Return compleTableFuture. allOf(task1,task2,task3).join(); return result;Copy the code

At first glance, this code is nothing special. Each task invokes an RPC request. The initial test of this code is to start the project with IDEA, that is, with SpringBoot embedded Tomcat, and this code works fine. And then the code starts to commit,merge.

After the next day, my colleague tested and found that this code threw an exception, and this function was the main entrance, so it was very blocked. At this time, I felt like this

When you look at the log in the background, you find that the exception was thrown by RPC internal processing. Your first response is to ask the upstream service provider if they have changed the interface. Get ready to shake the pot!

Then the result is no!! So I ran the next project, tested the interface, no problem! No problem! Oh my God?? Even weirder is that when you install several environments at the same time, the rest of the environment is fine, and then you don’t pay attention to it, only to find out that after you restart the server, this problem becomes a necessary problem, which is really a headache.

Problem orientation

Here can only honestly debug RPC call process source code. As shown in the code example, during the RPC call, the ServiceLoader will be used to find the corresponding implementation class of the XX interface. This configuration is in the RPC framework JAR package, which must be in the web-info /lib of the corresponding micro service.

The source code looks something like this:

       ArrayList list = new ArrayList<String>();
        ServiceLoader<T> serviceLoader = ServiceLoader.load(xxx interface);
        serviceLoader.forEach(xxx->{
            list.add(xxx)
        });
Copy the code

At the end of this step, if the list is empty, an exception will be thrown, which is the exception in the RPC call described above.

If you can’t load the ClassLoader, you need to suspect the ClassLoader

  • Bootstrap ClassLoader

Rt. jar, resources.jar, charsets.jar and class under %JRE_HOME%\lib

  • ExtClassLoader

Jar packages and classes in %JRE_HOME% lib\ext

  • AppClassLoader

Class in the path specified by the current application ClassPath

  • ParallelWebappClassLoader

This is a custom Tomcat ClassLoader that can load web-info /lib in the current application

Take a look at the ServiceLoader implementation:

public static <S> ServiceLoader<S> load(Class<S> service) { ClassLoader cl = Thread.currentThread().getContextClassLoader(); return ServiceLoader.load(service, cl); } public static <S> ServiceLoader<S> load(Class<S> service, ClassLoader loader) { return new ServiceLoader<>(service, loader); } private ServiceLoader(Class<S> svc, ClassLoader cl) { service = Objects.requireNonNull(svc, "Service interface cannot be null"); loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl; acc = (System.getSecurityManager() ! = null) ? AccessController.getContext() : null; reload(); }Copy the code

Loader = (cl == null); ClassLoader.getSystemClassLoader() : cl; , if the incoming this is null (null means BootStrapClassLoader), use this. GetSystemClassLoader (), is actually AppClassLoader.

When the serviceloader.load method is executed, what is the final loader for the ServiceLoader?

  • 1.Debug Applications that are started using Tomcat embedded in Sring Boot

In this case this is org. Springframework. Boot. Web. Embedded. Tomcat. TomcatEmbeddedWebappClassLoader

  • 2.Debug Applications that are started using Tomcat

In this case, the ClassLoader is AppClassLoader and null is obtained by thread.currentThread ().getContextClassLoader()

We’re getting closer to the truth. Why is the same code that the Tomcat application started to get the current context classloader for the thread that is the current context classloader?

. The problem is CompletableFuture runAsync here, there is no specified Executor, so will use ForkJoinPool thread pool, and ForkJoinPool threads in this will not inherit the parent thread. Enmm, amazing, why not inherit, I don’t know…

The problem to confirm

If the child thread inherits a ClassLoader from the parent thread, the child thread will inherit a ClassLoader from the parent thread.

class MyClassLoader extends ClassLoader{
    
}
Copy the code

Testing a

private static void test1(){ MyClassLoader myClassLoader = new MyClassLoader(); Thread.currentThread().setContextClassLoader(myClassLoader); New Thread(()->{system.out.println (thread.currentThread ().getContextClassLoader()); }).start(); }Copy the code

The output

classloader.MyClassLoader@4ff782ab
Copy the code

Test conclusion: If the child Thread is created by the common new Thread method, it will inherit the context ClassLoader of the parent Thread

* Source code analysis: Check the source code of new Thread to find the following code

        if (security == null || isCCLOverridden(parent.getClass()))
            this.contextClassLoader = parent.getContextClassLoader();
        else
            this.contextClassLoader = parent.contextClassLoader;

Copy the code

So the child thread’s context ClassLoader inherits from the parent thread’s context ClassLoader

Test two

Execute the following code in the Tomcat container environment

        MyClassLoader myClassLoader = new MyClassLoader();

        Thread.currentThread().setContextClassLoader(myClassLoader);

        CompletableFuture<Void> task1 = CompletableFuture.runAsync(() -> {
            System.out.println(Thread.currentThread().getContextClassLoader());
        });
Copy the code

The output

null
Copy the code

However, if the above code is executed through main, it will still print from the defined classloader

Why? Checked the information, Tomcat default SafeForkJoinWorkerThreadFactory as ForkJoinWorkerThreadFactory, then look at the SafeForkJoinWorkerThreadFactory source code

private static class SafeForkJoinWorkerThread extends ForkJoinWorkerThread { protected SafeForkJoinWorkerThread(ForkJoinPool pool) { super(pool); this.setContextClassLoader(ForkJoinPool.class.getClassLoader()); }}Copy the code

Here found that ForkJoinPool Settings. This is a Java thread util. Concurrent. ForkJoinPool class loader, and such in rt. The jar package, then it’s class loader is BootStrapClassLoader nature

Problem solving

Solution 1:

        ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader();

        CompletableFuture<Void> task1 = CompletableFuture.runAsync(() -> {
            Thread.currentThread().setContextClassLoader(contextClassLoader);
        });
Copy the code

The context ClassLoader is reset in the ForkJoinPool thread

Solution 2:

        CompletableFuture<Void> task1 = CompletableFuture.runAsync(() -> {
         
        },new MyExecutorService());
Copy the code

Instead of using the CompletableFuture’s default thread pool, ForkJoinPool, we use our custom thread pool

reference

Segmentfault.com/a/119000002… www.jianshu.com/p/8fcce16ae… Blog.itpub.net/69912579/vi…