The wide application of concurrent processing is the fundamental reason that Amdahl’s Law replaces Moore’s Law as the source of computer performance development, and it is also the most powerful weapon for human to squeeze computer computing power.

Major operating systems provide threading implementations, and the Java language provides uniform handling of threading operations across different hardware and operating system platforms. Each instance of the java.lang.Thread class that has called the start() method and has not yet terminated represents a Thread. The Thread class differs significantly from most Java library apis in that all of its key methods are declared Native. In Java library apis, a Native method often means that the method is not implemented or cannot be implemented using platform-independent means.

Implementation of threads

Thread is a more lightweight scheduling execution unit than process. The introduction of thread can separate the resource allocation and execution scheduling of a process. Each thread can share process resources (memory address, file I/O, etc.), and can schedule independently.

There are three main ways to implement threads: kernel threads (1:1 implementation), user threads (1: N implementation), and a mixture of user threads and lightweight processes (N: M implementation). The Threading model of the Java virtual machine is generally implemented based on the native threading model of the operating system, which uses a 1:1 threading model.

1. Kernel thread implementation

The implementation using kernel threads is also known as a 1:1 implementation. Kernel-level threads (KLT) are threads directly supported by the operating system Kernel. These threads are switched by the Kernel, which manipulates the Scheduler to schedule the threads. And is responsible for mapping the tasks of the thread to the individual processors. Each Kernel thread can be viewed as a doppelgant of the Kernel, so that the operating system can handle more than one thing at a time. A Kernel that supports multithreading is called a multi-threads Kernel.

2. User thread implementation

The way user threads are implemented is known as the 1: N implementation. Broadly speaking, a Thread as long as it’s not a kernel Thread, can be considered a User Thread a User Thread, (UT), so look from this definition, a lightweight process also belong to the User Thread, but the realization of lightweight process is always based on the kernel, should undertake many operating system calls, so efficiency could be limited, Does not have the usual benefits of user threads.

3. Hybrid implementation

An implementation that uses both kernel threads and user threads is called an N: M implementation. Under this hybrid implementation, there are both user threads and lightweight processes. User threads are still built entirely in user space, so user threads are still cheap to create, switch, and destruct, and can support large-scale user thread concurrency.

Java thread implementation

Threads of the Java virtual machine model based on the operating system native threads, each Java thread is directly mapped to an operating system native threads, and no additional indirect structure in the middle, so the Java virtual machine is not to interfere in thread scheduling (can set the thread priority to the operating system scheduling advice), It’s up to the underlying operating system to decide when to freeze or wake up a thread, how much processor execution time to allocate to the thread, and which processor core to assign to it.

The threading model supported by the operating system greatly influences how the threads of the Java virtual machine are mapped. The threading model only affects the concurrency scale and operation cost of a thread, and these differences are completely transparent to the code and execution of a Java program.

Java thread scheduling

Thread scheduling refers to the process in which the system allocates processor rights to threads. The thread scheduling method used in Java is preemptive scheduling. There are two main scheduling methods:

1. Cooperative Threads-scheduling:

In the multi-threaded system with cooperative scheduling, the execution time of the thread is controlled by the thread itself. After the thread completes its own work, it should inform the system to switch to another thread. The biggest advantage of cooperative multithreading is simple implementation, and because the thread to finish their own things will be thread switch, switch operation is known to the thread itself, so there is generally no thread synchronization problem. The downside is obvious: thread execution time is out of control, and even if a thread is not written properly and never tells the system to switch, the program will always block.

2. Preemptive threads-scheduling

In multithreaded systems with preemptive scheduling, each thread is allocated execution time by the system, and thread switching is not determined by the program itself. For example, in Java, the Thread::yield() method can yield execution time, but there is nothing the Thread can do to proactively obtain execution time. In this way, the execution time of the thread is controlled by the system, and there will be no problem that a thread will cause the whole process or even the whole system to block.

Java thread state

The Java language defines six thread states. At any point in time, a thread can have only one of these states and can switch between them in a specific way. The six states are:

New: A thread that has not been started since it was created is in this state. Runnable: Includes Running and Ready in the operating system thread state, where the thread may be executing or waiting for the operating system to allocate time for execution. Waiting: threads in this state are not allocated processor execution time; they wait to be explicitly woken up by another thread. The following methods cause the thread to be stuck in an indefinite wait state:

  • Object:: Wait () method with no Timeout parameter;
  • Thread::join() without Timeout;
  • LockSupport: : park () method.

Timed Waiting: Threads in this state are also not assigned processor execution time, but instead of Waiting to be explicitly awakened by another thread, they are automatically awakened by the system after a certain amount of time. The following methods cause the thread to enter the finite wait state:

  • Object:: Wait () with Timeout;
  • Thread::join() with Timeout;
  • LockSupport: : parkNanos () method;
  • LockSupport: : parkUntil () method.

Blocked: A thread is Blocked. The difference between a Blocked state and a wait state is that a Blocked state is waiting to acquire an exclusive lock, an event that occurs when another thread abandons the lock. A “waiting state” is a period of time waiting for a wakeup action to occur. The thread enters this state while the program is waiting to enter the synchronization zone. Terminated: The thread state of a Terminated thread. Terminated execution is Terminated.

The six states mentioned above will transition to each other when certain events occur.

Thread state transition relationship

Java thread priority

Although Java thread scheduling is automated, you can still “suggest” that the operating system allocate more execution time to some threads and less to others — this is done by setting thread priorities.

The Java language sets 10 levels of Thread priority (thread.min_priority through thread.max_priority). When two threads are in the Ready state at the same time, the thread with higher priority is more likely to be selected by the system for execution. However, thread priority is not a stable adjustment, and it is clear that thread scheduling is ultimately up to the operating system because Java threads on mainstream virtual machines are mapped to the system’s native threads. Although most modern operating systems provide the concept of thread priority, it does not necessarily correspond to the priority of Java threads.

Java daemon thread

There are two types of threads: regular threads and daemon threads. Of all threads created at JVM startup, except for the main thread, the other threads are daemons (such as garbage collectors and other threads that perform auxiliary work). You want to create a thread to do some auxiliary work, but you don’t want that thread to block the JVM from shutting down. In this case, Daemon threads are needed.

Public static void main(String[] args) {thread.setdaemon (true) {thread.setdaemon (true) new Thread(new DaemonRunner(), "DaemonRunner"); Daemon properties need to be set before starting a thread, not after. thread.setDaemon(true); thread.start(); } static class DaemonRunner implements Runnable { @Override public void run() { try { System.out.println("run... ") ); } finally {// System.out.println("DaemonThread finally run."); }}}Copy the code

When a new thread is created, the new thread inherits the daemon state of the thread that created it, so by default, all threads created by the main thread are normal threads. The only difference between a normal thread and a daemon thread is what happens when the thread exits. When a thread exits, the JVM checks the other running threads, and if they are daemons, the JVM exits normally. When the JVM stops, any remaining daemons are discarded — not necessarily executing a finally code block. Daemons should be used as little as possible — very few operations can be safely discarded without cleaning up. In particular, it can be dangerous to perform tasks that may involve I/O operations in a daemon thread. Daemon threads are best used to perform “internal” tasks, such as periodically removing overdue data from an in-memory cache.

Thread overhead

Single-threaded programs have neither thread scheduling nor synchronization overhead, and do not require locks to ensure consistency of data structures. There are performance costs associated with scheduling and coordination of multiple threads: for threads introduced to improve performance, the performance gains from parallelism must outweigh the costs of concurrency.

Context switch

If the main thread is the only thread, it is almost never scheduled. On the other hand, if the number of runnable threads is greater than the number of cpus, the operating system will eventually schedule out one of the running threads so that other threads can use the CPU. This causes a context switch in which the execution context of the current running thread is saved and the execution context of the newly scheduled thread is set to the current context.

Context switching requires some overhead, while thread scheduling requires access to data structures shared by the operating system and JVM. Applications, operating systems, and JVMS all use the same set of cpus. But the overhead of context switching is not just the overhead of including the JVM and operating system. When a new thread is switched in, the data it needs may not be in the current processor’s local cache, so the context switch will result in some cache misses, and the thread will run more slowly on the first schedule. This is why the scheduler allocates a minimum execution time for each runnable thread, even if there are many other threads waiting to execute: it spreads the overhead of context switching over more execution time that will not break, thus increasing overall throughput (at the expense of responsiveness). When a thread is blocked waiting for a contended lock, the JVM typically suspends the thread and allows it to be swapped out. If threads block frequently, they will not be able to use the full schedule slice. The more blocking that occurs in a program (including blocking I/O, waiting to acquire contending locks, or waiting on condition variables), the more context switches that occur with CPU-intensive programs, increasing scheduling overhead and thus reducing throughput. The actual overhead of context switching varies from platform to platform, and on most general-purpose processors it can be a matter of microseconds.

blocking

Competing synchronization may require operating system intervention, increasing overhead. When a contention occurs on a lock, the thread that lost the contention is bound to block. The JVM can implement blocking behavior either by spin-waiting (iterating through attempts to acquire the lock until it succeeds) or by suspending the blocked thread through the operating system. The efficiency of these two approaches depends on the overhead of context switching and the time required to wait before the lock is successfully acquired. If the wait time is short, the spin wait mode is suitable, while if the wait time is long, the thread suspend mode is suitable. Some JVMS will choose between the two based on historical wait time analysis data, but most JVMS simply suspend threads while waiting for locks. The thread needs to be suspended when it is unable to acquire a lock or because it is waiting on a condition or blocked on an I/O operation, which involves two additional context switches and all the necessary operating system operations and caching operations: Blocked threads are swapped out before their execution slice runs out, and switched back again later when locks or other resources to be acquired become available. (When blocking due to lock contention, the thread has some overhead in holding the lock: when it releases the lock, it must tell the operating system to resume running the blocked thread.)

Risk of thread

Security issues

Thread safety can be very complex, and without adequate synchronization, the order in which operations are performed in multiple threads can be unpredictable and can even produce strange results. Because multiple threads share the same memory address space and are running concurrently, they may access or modify variables that are being used by other threads. Of course, this is a great convenience, because it is much easier to share data in this way than other inter-thread communication mechanisms. But it also comes with a huge risk: threads can go wrong due to unexpected changes in data. When multiple threads simultaneously access and modify the same variable, non-serialization factors are introduced into the serial programming model, and this non-serialization is difficult to analyze. For a multithreaded program to behave predictably, access to shared variables must be coordinated so that threads do not interfere with each other. Fortunately, Java provides various synchronization mechanisms to coordinate this access.

Activity problem

In serial programs, one form of the activity problem is an unintentional infinite loop that makes it impossible to execute code after the loop. Threads introduce other activity issues deadlocks, starvation, and live locks. Like most concurrency errors, errors that cause activity problems are also difficult to analyze because they depend on the timing of events in different threads and therefore cannot always be reproduced during development or testing.

A deadlock

When one thread holds a lock forever, and other threads try to acquire it, they are blocked forever. While thread A holds lock L and wants to acquire lock M, thread B holds lock M and tries to acquire lock L, the two threads will wait forever. This is the simplest form of deadlock (or “DeadlyEmbrace”), in which multiple threads wait forever because of a loop’s lock dependency.

hunger

Starvation occurs when a thread cannot continue execution because it does not have access to the resources it needs. The most common source of hunger is the CPU clock cycle. If the priority of threads is misused in Java applications, or if some unfinishable structure is performed while holding the lock (such as an infinite loop, or waiting indefinitely for a resource), starvation can also occur because other threads that need the lock will not be able to get it.

Live lock

Livelock is another form of an activity problem that, while it does not block, cannot continue because the thread will do the same thing over and over again, always failing. Live locks typically occur in applications that process transaction messages: if a message cannot be successfully processed, the message processing mechanism rolls back the entire transaction and puts it back at the top of the queue. If a message handler has an error in processing a particular type of message that causes it to fail, transaction rollback occurs every time the message is fetched from the queue and passed to the handler with the error. Since the message is put back at the beginning of the queue, the handler is called repeatedly and returns the same result. Although the thread processing the message is not blocked, it cannot continue. This form of live lock is usually caused by excessive error recovery code, because it mistakenly treats unfixable errors as fixable errors.

Thread safety

When multiple threads access to an object at the same time, if don’t have to consider these threads in the runtime environment of scheduling and execution alternately, also do not need to undertake additional synchronization, or any other coordinated operation in the caller, call the object’s behavior can get the right results, it is said that an object is thread-safe.

It requires thread-safe code must have a common feature: the code itself encapsulates all the necessary correctness guarantee means (such as mutual exclusion synchronization, etc.), so that the caller does not need to care about the call problem in multi-threaded environment, let alone implement any measures to ensure the correct call in multi-threaded environment.

Thread safety in Java

The data shared by various operations in the Java language is divided into five categories: immutable, absolute thread-safe, relative thread-safe, thread-compatible, and thread-antagonistic.

1. Immutable: Objects that are Immutable are thread-safe, requiring no thread-safety safeguards either for their method implementation or for their callers. In the Java language, if the data shared by multiple threads is a basic data type, it is guaranteed to be immutable as long as it is defined with the final keyword. An object is immutable if:

  • The state of an object cannot be changed after it is created.
  • All fields of the object are of final type
  • Object created correctly (this reference is not selected during object creation)

2. Absolute thread safety: callers don’t need any additional synchronization measures regardless of the runtime environment

3. Relative thread safety: Relative thread safety is what we generally speaking thread-safe, it need to make sure that the object of a single operation is thread-safe, we do not need additional when calling to safeguard measures, but for some particular sequence of consecutive calls, may need to end the call with additional synchronization method to guarantee the correctness of the call.

4. Thread-compatible: Thread-compatible means that the object itself is not thread-safe, but can be safely used in a concurrent environment by properly using synchronization on the calling side.

5. Thread opposition: Thread opposition is the inability to use code concurrently in a multithreaded environment, regardless of whether the caller has taken synchronization measures. Because of the Java language’s natural support for multithreading, thread-opposition code that excludes multithreading is rare, often harmful, and should be avoided as much as possible.

Thread-safe implementation

1. Mutual Exclusion & Synchronization

Mutex Synchronization, also known as Blocking Synchronization, is the most common and primary concurrency correctness guarantee. Synchronization refers to ensuring that shared data is used by only one (or a few, when using semaphores) thread at a time when multiple threads concurrently access the data. Mutex is a means to realize synchronization. Critical Section, Mutex and Semaphore are common ways to realize Mutex. Therefore, in the word “mutually exclusive synchronization”, mutual exclusion is the cause, synchronization is the effect; Mutual exclusion is the method, synchronization is the destination. In Java, mutex is synchronized and ReentrantLock.

2. Non-blocking synchronization

Mutex synchronization is the main problem for thread blocking and awakening to the performance overhead, mutex synchronization belongs to a kind of pessimistic concurrency strategy, no matter whether Shared data there will be a competition, it will add lock (discussed here is a conceptual model, in fact, virtual opportunity to optimize away a large part of unnecessary lock), This leads to the overhead of user-to-core mindset transitions, maintaining lock counters, and checking if there are blocked threads that need to be woken up.

With the development of hardware instruction sets: replacing locks with underlying atomic machine instructions (such as compare and swap instructions) to ensure consistency of data over concurrent access. Implementation of this optimistic concurrency strategy eliminates the need to block and suspend threads, so such Synchronization operations are called non-blocking Synchronization, and code that uses this measure is often referred to as lock-free programming. Starting with Java5.0, you can use atomic variable classes, such as AtomicInteger and AtomicReference, to build efficient non-blocking algorithms.

3. No synchronization scheme is available

Neither blocking nor non-blocking synchronization is necessary to be thread-safe, and synchronization is not necessarily related to thread-safety. Synchronization is only a means to ensure the correctness of shared data contention. If a method does not involve shared data in the first place, it does not need any synchronization measures to ensure its correctness. One way to avoid synchronization is to not share data. The technique is called ThreadConfinement, and it’s one of the simplest ways to achieve thread-safety. Java implements thread-local storage through the java.lang.ThreadLocal class.

Being able to write high performance, scalable concurrent programs is an art, but understanding how concurrency is implemented at the bottom of the system is a prerequisite for mastering this art

reference

An in-depth understanding of the Java Virtual Machine: Advanced JVM features and Best Practices (version 3) Java Concurrent Programming hands-on