• Internal Java memory model
  • Memory model at the hardware level
  • The connection between Java memory model and hardware memory model
  • Visibility of shared objects
  • Resources racing

The Java memory model is a good example of how the JVM works in memory. The JVM can be understood as an operating system executed by Java. As an operating system, there is a memory model, which is often referred to as the Java memory model.

If we want to write multithreaded parallel programs correctly. It is important to understand how the Java memory model works in multithreading to better understand the underlying workings.

The Java memory model describes how and when different threads can see other threads write values of shared variables, and how synchronizers share variables. The original Java memory model was not good enough and had many shortcomings, so in Java 1.5z, the version of the Java memory model received a major update and improvement, and is still used in Java 8.

Internal Java memory model

The JVM’s internal memory model is divided into two parts, thread stack and heap. The complex memory model is abstracted as follows:

Each thread running in the JVM has its own thread stack in memory. A thread stack typically contains information about where the thread’s methods have been executed. Also known as the “call stack,” when a thread executes code, the call stack changes with the state of execution.

The thread stack also contains local variables for each method execution, and all methods are stored on the thread stack. A thread can access only its own thread stack. Local variables created by each thread are not visible to other threads, that is, private. Even if two threads call the same method, each thread keeps a copy of the local variable, which belongs to its own thread stack.

All local variables of the basic type (Boolean, byte, short, char, int, long, float, double) are stored in the thread stack and are not visible to other threads, One thread may pass a copy of the value of a primitive variable to another thread, but its own variables cannot be shared, only copies can be passed.

The heap stores objects that are new from Java programs, regardless of which thread they are new from, all together, regardless of which thread they belong to. These objects also contain versions of objects of primitive types (e.g. Byte, Integer, Long etc.). Whether the object is assigned to a local variable or a member variable, it ends up in the heap.

The thread stack holds the local variable and the heap holds the object.

A local variable of a primitive data type will be stored entirely in the thread stack.

A local variable can also be a reference to an object, in which case the local variable resides on the thread stack, but the object itself resides on the heap.

An object may contain methods that also contain local variables that are stored on the thread stack, even though the objects and methods they belong to are stored on the heap.

An object’s member variables are stored on the heap along with the object itself, regardless of whether the member variables are raw data types or references to the object.

Static class variables are also typically stored on the heap, depending on the class definition.

Objects stored on the heap can be accessed by all threads through references. When a thread holds a reference to an object, it also has access to the object’s member variables. If two threads call a method on the same object at the same time, they will both own member variables of the object, but each thread will have its own private local variables.

The graph below illustrates the above

Both threads have a set of local variables. One of the Local variables (Local Variable 2) points to Object3 in the heap. The two threads each have a different reference to the same object object3. Their references are local variables and exist in their own thread stacks, even though the two different references refer to the same object.

We can also see that the shared object object3 has references to Object2 and object4 that exist as member variables in Object3. Object2 and Object4 are accessible to both threads by reference to member variables in Object3.

The diagram also illustrates local variables that refer to different objects in the heap. For example, object1 and object5 in the figure are not the same object. In theory, any thread can access an object in the heap, as long as the thread holds a reference to the object in the heap. But in this diagram, each thread has only one reference to one of these two objects.

Now, we’ll write an actual piece of code with a memory model that looks like this:

public class MyRunnable implements Runnable() {

    public void run() {
        methodOne();
    }

    public void methodOne() {
        int localVariable1 = 45;

        MySharedObject localVariable2 = MySharedObject.sharedInstance; / /...do more with local variables.

        methodTwo();
    }

    public void methodTwo() {
        Integer localVariable1 = new Integer(99); / /...do more with local variable.
    }
}
Copy the code
public class MySharedObject {

    //static variable pointing to instance of MySharedObject

    public static final MySharedObject sharedInstance =
        new MySharedObject();


    //member variables pointing to two objects on the heap

    public Integer object2 = new Integer(22);
    public Integer object4 = new Integer(44);

    public long member1 = 12345;
    public long member1 = 67890;
}
Copy the code

If two threads execute the run method, the memory model shown above is the result of the program.

MethodOne () declares a local variable of the original data type, localVariable1 of type int, and a local variable of a reference to the object (localVariable2).

Each thread that executes methodOne() creates its own copy of the local variables, localVariable1 and localVariable2, in the space of their thread stack. LocalVariable1 will be completely invisible to other threads, existing only with each thread in its own thread stack space. One thread cannot see changes and operations made by other threads to localVariable1, so it is invisible.

Each thread executing methodOne() also creates a copy of localVariable2, but different copies of localVariable2 end up pointing to objects on the same heap. This code makes localVariable2 point to the object previously referenced by a static variable. There is only one copy of a static variable, there are no extra copies, and static variables are stored in the heap. So, both copies of localVariable2 point to an instance of the same MySharedObject object, as well as a static variable in the heap that points to the same object instance. This object corresponds to Object3 in the figure above.

We found that MySharedObject contains these two member variables. These member variables are stored on the heap like objects. These two member variables point to two INTEGER objects. These two objects correspond to Object2 and Object4 respectively in the figure above.

We see that methodTwo() creates a local variable called localVariable1. This local variable is a reference to an object that points to an INTEGER object. This method points the local variable localVariable1 to a new value. When methodTwo() is executed, each thread holds a copy of localVariable1. The two Integer objects will be initialized on the heap, but because the method creates a new object each time it executes, the two threads will have separate object instances. These two objects correspond to Object1 and Object5 in the figure above.

We find that the member variables in MySharedObject are primitive data types, but since they are member variables, they are still stored on the heap. Only local variables are stored in the thread stack.

Memory model at the hardware level

The memory structure at the hardware level is different from the memory structure in the JVM. For us, it is necessary to understand the memory model at the hardware level correctly, which can help us understand the underlying mechanism of Java multithreading, but also to understand how the Java memory model works on the hardware memory structure. This chapter covers the hardware-level memory model, and the next section covers how Java works with hardware.

Here is a simplified diagram of modern computer hardware:

Modern computers typically have two or more cpus, and these cpus may also have multiple cores. This means that a computer with multiple cpus may have multiple threads running at the same time, with each CPU running one thread at any given time. This means that if our Java program is multithreaded, internally each thread will have one CPU running at the same time.

Registers Are the registers that each CPU has in its memory. These registers are important. It is much faster for a CPU to perform calculations in registers than in main memory. This is because the CPU can access registers much faster than it can access memory.

Each CPU also has one CPU cache. This is because the CPU accesses the cache much faster than it accesses the memory, but slower than it accesses the registers, so the speed of the cache is somewhere between register and memory. Some cpus also have multiple levels of cache, such as (Level 1 and Level 2), but this is irrelevant to our understanding of the Java memory model. We only need a three-tier memory structure, register-cache-memory (RAM).

A computer usually has main memory, or RAM, which is accessible to all cpus and is usually much larger than the cache.

Generally, when the CPU needs to access memory, it will first read a part of the main memory to the cache, or even read a part of the cache to the internal register, and then perform calculations in the register. When the CPU writes the results back to memory, it flushes the data in the registers and cache, and writes the values back to memory.

When the CPU asks the cache to store something else, the cache is flushed into memory. The CPU cache can write some data to the memory while writing some data to its own cache. Therefore, it is unnecessary to empty the cache when updating data. You can read and write data at the same time. Typically, the cache actually updates data on smaller chunks of memory, called “cache lines.” Multiple “cache lines” may be reading data into the cache, while another may be writing data back into memory.

The connection between Java memory model and hardware memory model

As mentioned above, the Java memory model is different from the hardware memory model. The hardware memory model does not distinguish between heap and stack. At the hardware level, all thread stacks and heaps are stored in main memory, and a portion of thread stacks and heaps may sometimes appear in CPU caches and CPU registers. The following figure illustrates this problem:

When objects and variables are stored in different areas of memory, many problems can occur. There are two main types of problems:

  • Visibility issues when threads update or write some shared data
  • Resource racing is a problem when reading and writing shared data, both of which will be discussed in the next section

Visibility of shared objects

If multiple threads share an object and do not use the volatile or Synchronize declaration correctly, updating the shared object can be a problem that is invisible to other threads.

We assume that the shared object initializes in main memory. A thread running in the CPU reads the shared object into the cache. At this point, the shared object may change as the program executes. As long as the CPU’s cache has not been written back to main memory, the changes to this shared object are not visible to other threads running on the CPU. In this case, each thread holds its own copy of the shared object, which is stored in the cache of its own CPU and is not visible to other threads.

To illustrate the general situation, the CPU thread on the left reads the shared object into the cache and changes its value to 2. This change is not visible to the other threads of the CPU on the right, because the update to the variable count has not yet been written back into main memory.

Want to solve the problem of the Shared object visibility, you can use Java’s volatile keyword (see the author’s another volatile post), the keyword can be guaranteed by the given variables are read directly from the main memory, and when the update immediately write back to memory, so you can ensure that the change is visible in time.

Resources racing

If multiple threads share an object, and multiple threads need to update variables in the shared object, then resource racing can occur.

Let’s say thread A reads the variable count from A shared object into the CPU’s cache. Thread B does the same thing, but reads it from A different CPU’s cache. Now thread A increments count by one, and thread B does the same thing. Each in the cache of a different CPU.

If the increments are performed sequentially, the variable count is incremented twice and by 2 from its original value, written back to main memory.

However, if the increments are performed concurrently and are not synchronized properly, the updated value will only be incremented by one when written back to memory, even though the increments were actually done twice. The following diagram illustrates the problem of resource racing when programs are executed concurrently:

To solve this problem, use the Synchronize keyword in Java. Synchronize ensures that only one thread can enter a code section declared as synchronize. A synchronized thread ensures that all variables in the synchronized code block are read from memory, and that when the thread leaves the block, all updated values are written back to main memory, whether or not the variable is volatile.

summary

This paper analyzes the Java memory model and the hardware level memory model in detail, and analyzes how hardware and Java cooperate in the memory model. This is extremely important and provides a solid foundation for understanding the concept of multithreading in Java.