I’ve had the weirdest problem in six months

background

Recently, our company bought a set of projects, and a series of strange problems appeared in the startup process. The other party’s technology stack requires Tomcat7 to start, but our company requires Tomcat9 to start for security reasons.

Problem description

The following cases are the same WAR package and Tomcat

Due to my unfamiliarity with the project, IT took me a long time to find out the reason. During the search, Arthas was used to compile the problem class at runtime and found that the two classes came from different JARS, so the question turned to what was causing the loading order of the JARS.

Problem probe

Two classes with the same pathname and the same name are loaded only once by the class loader

JVM class loading is a tree structure. The higher the JVM hierarchy is, the earlier the class loader will load the classes in its path. Here is the level of Tomcat’s classloader.

      Bootstrap
          |
       System
          |
       Common
       /     \
  Webapp1   Webapp2 .

Copy the code

We know that the two jars in question are in the same class loader, so we rule out problems caused by different levels of class loaders.

Principles of Tomcat7 Loading Jar packages

Tomcat implements its own class loader, which is used to load all the class files in the JAR package of its local project. Therefore, under the same class loader, if the path name and class name are the same, the loading order is determined by the order of the JAR package. The class whose JAR package comes first will be loaded first.

But why does everything work in Tomcat7 and not in Tomcat9? So I looked at the source code for Tomcat7 when the Context loaded the jar package in the project

Tomcat7 load jar part in WebappLoader. SetRepositories () method, paste out some important code.

 // Looking up directory /WEB-INF/lib inthe context NamingEnumeration<NameClassPair> enumeration = null; Enumeration = libdir.list ("");
    } catch (NamingException e) {
        IOException ioe = new IOException(sm.getString(
                "webappLoader.namingFailure", libPath));
        ioe.initCause(e);
        throw ioe;
    }

Copy the code

List is the path to all jar packages in the web-INF lib of the application. We can trace it in and see that the list method of FileDirContext has the following sentence

Arrays.sort(names);             // Sort alphabetically

Copy the code

We can see that in Tomcat7 there is a sort action to get all the jar packages. The jar packages are sorted by the first letter A-Z. The jar we want to load starts with the wrong jar.

Principles of Tomcat9 Jar package Loading

The reason why Tomcat7 works in all projects is because Tomcat7 does the sort action. What does Tomcat9 do when it loads the Jar?

Tomcat9 when loading source is through StandardRoot processWebInfLib () method to load

    protected void processWebInfLib() throws LifecycleException {
        WebResource[] possibleJars = listResources("/WEB-INF/lib".false);
        for (WebResource possibleJar : possibleJars) {
            if (possibleJar.isFile() && possibleJar.getName().endsWith(".jar")) {
                createWebResourceSet(ResourceSetType.CLASSES_JAR,
                        "/WEB-INF/classes", possibleJar.getURL(), "/"); }}}Copy the code

Here we can see that Tomcat doesn’t do anything with the Jar, just File File = new File(). So why does the same Tomcat9 and same War package start up in Windos, but not in macOS or Linux? After testing, we found that all files under the Java get folder are related to the file system of the operating system. If you pull out the same folder contents in Windows, you will find that the output is sorted a-Z. But on macOS or Linux you can print the natural order with the command lL-fi, and you’ll see no pattern.

To solve

Now that we’ve explained all of the problems described above, it’s time to figure out how to solve them.

1. Modify the source code for Tomcat9 to sort all Jar packages when they are retrieved
2. Resolve conflicting files

The first solution only solves the problem for a while, that is, the project gets started properly, but once subsequent changes are made to related classes, which classes are in conflict? Then this problem must be a time bomb.

The second solution is to find the files with conflicts, and then find out the ones that are not used and delete them. However, when one is deleted, others will pop up. After deleting several, I find that the codes of the purchased projects are not standardized, so this phenomenon is very common. I wrote a script to run out of all the files of the same class in the project.

The script idea

1. Find all the Java files
2. Go to the Java filepackageThat row, and then read that row
3. packageThe following package name is concatenated with the class name into the List collection
4. Filter out the same content in the collection

The script code can be found on GitHub. Using simple instructions, put all the project codes you want to scan in one folder, for example, I want to scan A, B, C, and D.

--/
  --scanDir
   --A
   --B
   --C
   --D

Copy the code

All I have to do is import the Jar package and call it as follows

List<String> list = FindDuplicate.findDuplicatePath("/scanDir/");

Copy the code

Return is a collection of a record says there is a group of conflicting files, two conflicting files path is | | | | | | | |

Script code

Previous articles on Tomcat

• A weird trip to find StackOverflowError problems
• How to debug Tomcat source code breakpoint
• Tomcat Series 1 — Overall architecture