GitHub github.com/wangzhiwubi…
Big Data into the Road of God series:
GitHub github.com/wangzhiwubi…
Java Advanced Feature Enhancements – Collection
Java Advanced Feature enhancements – Multithreading
Java advanced features enhancements -Synchronized
Java Advanced feature enhancements – Volatile
Java Advanced Feature Enhancements – Concurrent Collections framework
Java Advanced Features enhancement – Distributed
Java Advanced Feature enhancements -Zookeeper
Java Advanced Features enhancements -JVM
Java Advanced Feature Enhancements -NIO
The public,
-
The only net from 0 to help Java developers to do big data domain public number ~
-
Http://import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata/import_bigdata
Java Advanced Features Enhancements – locking
This part of the network has a large number of resources can refer to, in this part of the collation, thank the predecessors pay, at the end of each section of the article has a reference list, source code recommendation see JDK1.8 later version, pay attention to the screen ~ #### multithreading ### collection framework #### #Java concurrent container
The lock
Lock classification in Java
There are many concurrent articles that mention various locks such as fair locks, optimistic locks, etc. This article introduces various locks. The introduction is as follows: Fair lock/unfair lock Reentrant lock Exclusive lock/shared lock Mutex lock/read/write lock Optimistic lock/Pessimistic lock segmentalized lock biased lock/lightweight lock/heavyweight lock spin lock The following is a summary of the definitions of each lock. Fair lock/Unfair lock FAIR lock means that multiple threads acquire locks in the order in which they apply for locks. An unfair lock means that multiple threads obtain locks in a different order than the one that applied for the locks earlier. Possibly, it could lead to priority reversals or starvation. In the case of Java ReentrantLock, the constructor specifies whether the lock is fair, and the default is unfair. The advantage of an unfair lock is that the throughput is greater than that of a fair lock. Synchronized is also an unfair lock. Since it does not implement thread scheduling via AQS like ReentrantLock, there is no way to make it a fair lock.
Reentrant lock A reentrant lock, also known as a recursive lock, means that the inner method automatically acquires the lock when the same thread acquires the lock in the outer method. A bit abstract, here is an example of the code. In the case of Java ReentrantLock, its name can be seen as a ReentrantLock. Its name is ReentrantLock. For Synchronized, it is also a reentrant lock. One benefit of reentrant locks is that deadlocks are somewhat avoided.
synchronized void setA() throws Exception{
Thread.sleep(1000);
setB();
}
synchronized void setB() throws Exception{
Thread.sleep(1000);
}
Copy the codeThe above code is a feature of a reentrant lock. Without a reentrant lock, setB may not be executed by the current thread, possibly causing a deadlock.
Exclusive lock/Shared lock Exclusive lock means that the lock can only be held by one thread at a time. A shared lock means that the lock can be held by multiple threads.
In the case of Java ReentrantLock, this is the exclusive lock. But ReadWriteLock, another implementation of Lock, has a shared read Lock and an exclusive write Lock. The shared lock of read lock ensures that concurrent read is very efficient. Read, write, read, and write processes are mutually exclusive. Exclusive lock and shared lock is also achieved through AQS, through the implementation of different methods, to achieve exclusive or shared. In the case of Synchronized, of course, the lock is exclusive.
Mutex/read-write lock The exclusive/shared lock mentioned above is a broad term, and the mutex/read-write lock is a concrete implementation. ReentrantLock in Java is ReadWriteLock
Optimistic locks/Pessimistic locks Optimistic locks and pessimistic locks do not refer to specific types of locks, but to the perspective of concurrent synchronization. Pessimistic locks believe that concurrent operations on the same data must be modified, even if no modification, it will be considered modified. Therefore, for concurrent operations on the same data, pessimistic locking takes the form of locking. The pessimistic view is that concurrent operations without locking are bound to cause problems. Optimistic locks assume that concurrent operations on the same data will not change. When the data is updated, the data will be updated by trying to update and constantly re – updating. Optimistically, concurrency without locking is fine.
From the above description, we can see that pessimistic lock is suitable for the scenario with a lot of write operations, optimistic lock is suitable for the scenario with a lot of read operations, not locking will bring a lot of performance improvement. Pessimistic locking is used in Java to take advantage of various locks. Optimistic locking is used in Java, is lockless programming, often using CAS algorithm, a typical example is atomic class, through CAS spin atomic operation update.
Segmented lock A segmented lock is actually a lock design, not a specific lock. For ConcurrentHashMap, its concurrent implementation is to implement efficient concurrent operations in the form of segmented locks. ConcurrentHashMap Segment lock, which is similar to the structure of HashMap (JDK7 and JDK8) That is, there is an internal array of entries, each element in the array is a linked list; It is also a ReentrantLock (the Segment inherits ReentrantLock). When a put element is required, instead of locking the entire HashMap, hashCode first knows which segment it is to be placed in, and then locks that segment. So when multiple threads put elements, as long as they are not placed in a segment, they achieve true parallel inserts. However, in order to obtain the global information of the HashMap, we need to obtain all the segment locks to obtain the statistics. Segment locking is designed to refine the granularity of the lock. When an operation does not need to update the entire array, only one item in the array is locked.
Biased lock/lightweight lock/heavyweight lock these three types of locks refer to lock states and are specific to Synchronized. In Java 5, efficient Synchronized is implemented by introducing a mechanism of lock escalation. The status of these three locks is indicated by an object monitor field in the object header. Biased locking is when a piece of synchronized code is always accessed by a thread, and that thread automatically acquires the lock. Reduce the cost of acquiring locks. Lightweight lock refers to that when a biased lock is accessed by another thread, the biased lock will be upgraded to lightweight lock. Other threads will try to acquire the lock through the form of spin, which will not block and improve performance. Heavyweight lock means that when the lock is a lightweight lock, the other thread spins, but the spin will not continue. When the spin is a certain number of times, the lock has not been acquired, it will enter the block, and the lock expands to the heavyweight lock. Heavyweight locks can block other threads that apply for them and degrade performance.
Spin lock In Java, a spin lock means that the thread attempting to acquire the lock does not block immediately, but instead attempts to acquire the lock in a circular manner. This has the advantage of reducing the consumption of thread context switching, but has the disadvantage of consuming CPU.
The Lock interface
Before the Lock interface, Java programs relied on the synchronized keyword to implement locking. After JDK1.5, the Lock interface and related implementation classes are added to the package to realize the Lock function.
While the scope mechanism of synchronized methods and statements makes it easier to program with monitor locks and helps avoid many common programming errors involving locks, sometimes you need to handle locks in a more flexible way. For example, some algorithms for traversing concurrent access data structures require the use of “manual” or “chain locking” : you get the lock on node A, then you get node B, then you release A and get C, then you release B and get D, and so on. The synchronized keyword is not as easy to implement in this scenario, and the Lock interface is much easier to use.
The Lock of the interface implementation class: already, ReentrantReadWriteLock ReadLock, ReentrantReadWriteLock. WriteLock
AbstractQueuedSynchronizer
When you look at the source code, you’ll be surprised to find that ReentrantLock doesn’t have much code, and one other feature that is obvious is: Basically all the realization of the method are actually call the static memory class methods in Sync, and Sync class inherits the AbstractQueuedSynchronizer (AQS). You can see to understand key core is already on the queue synchronizer AbstractQueuedSynchronizer (hereinafter referred to as synchronizer) understanding.
In the realization of the synchronization component, AQS is the core part. The implementer of the synchronization component realizes the semantics of the synchronization component by using the template method provided by AQS, and AQS realizes the management of the synchronization state, as well as the queuing of the blocking thread, waiting for the notification and so on. The core of AQS also includes these aspects: synchronous queue, exclusive lock acquisition and release, shared lock acquisition and release as well as interruptible lock, timeout wait lock acquisition of these features, and these are in fact template methods provided by AQS, summarized as follows: exclusive lock:
Void acquire(int ARg): Obtain the synchronization status exclusively. If the acquisition fails, insert the synchronization queue and wait. Void acquireInterruptibly(int ARg): Same as acquire, but can detect interrupts while waiting in a synchronous queue; Boolean tryAcquireNanos(int ARg, Long nanosTimeout): Wait for acquireInterruptiblyfalse; Boolean release(int arg): Releases synchronization state. This method wakes up the next node in the synchronization queueCopy the codeShared lock:
Void acquireShared (int arg) : Shared access to sync, and exclusive distinction lies in the same time have multiple threads access sync void acquireSharedInterruptibly (int arg) : Boolean tryAcquireSharedNanos(int arg, long nanosTimeout): On the basis of acquireSharedInterruptibly increased the timeout waiting for the function of the Boolean releaseShared (int arg) : Shared release syncCopy the codeReentrantLock
ReentrantLock ReentrantLock is a class that implements the Lock interface and is frequently used in practical programming. It supports reentrancy, indicating that it can repeatedly Lock shared resources, that is, the current thread will not be blocked when it obtains the Lock again. Java keyword synchronized implicitly supports reentrancy. Synchronized realizes reentrancy by acquiring autoincrement and releasing autodecrement. ReentrantLock also supports both fair and unfair locking. To fully understand ReentrantLock, it is necessary to learn the synchronization semantics of ReentrantLock. The realization principle of reentrancy; 2. Fair and unfair locks.
How reentrancy works
In order to support reentrancy, two problems need to be solved: 1. When a thread acquires a lock, if the thread that has acquired the lock is the current thread, it will immediately acquire the lock again. 2. Because the lock will be acquired n times, the lock will be released successfully only after it is released n times. From this article, we learned that the synchronization component expresses its synchronization semantics primarily by overriding several protected methods of AQS. For the first problem, let’s look at how ReentrantLock is implemented. Using an unfair lock as an example, we can determine whether the current thread can acquire the lock. The core method is nonfairTryAcquire:
final boolean nonfairTryAcquire(int acquires) { final Thread current = Thread.currentThread(); int c = getState(); //1. If the lock is not held by any thread, the lock can be acquired by the current threadif (c == 0) {
if (compareAndSetState(0, acquires)) {
setExclusiveOwnerThread(current);
return true; }} //2. If occupied, check whether the occupied thread is the current threadelse if(current == getExclusiveOwnerThread()) { // 3. Int nexTC = c + acquires;if (nextc < 0) // overflow
throw new Error("Maximum lock count exceeded");
setState(nextc);
return true;
}
return false;
}
Copy the codeThe logic of this code is also very simple, see the comments for details. In order to support reentrant, the processing logic is added in the second step. If the lock has already been held by a thread, it will continue to check whether the thread holding the lock is the current thread. If so, the synchronization status plus 1 returns true, indicating that the lock can be acquired again. Each refetch increments the synchronized state by one, so what happens when you free it? (Again using the unfair lock example) The core method is tryRelease:
protected final boolean tryRelease(int releases) { //1. Synchronization state minus 1 int c = getState() -releases;if(Thread.currentThread() ! = getExclusiveOwnerThread()) throw new IllegalMonitorStateException(); boolean free =false;
if(c == 0) {//2. The lock is released successfully only when the synchronization status is 0true
free = true;
setExclusiveOwnerThread(null); } // 3. Lock not fully released, returnfalse
setState(c);
return free;
}
Copy the codeNote that reentrant locks are not released until the synchronization state is 0, otherwise the lock is still held. If the lock was acquired n times and released n-1 times, false is returned if the lock was not fully released, and true is returned if the lock was released n times. At this point we can clarify the implementation of ReentrantLock ReentrantLock, which is the first step in understanding synchronization semantics.
Fair and unfair locks
ReentrantLock supports two types of locks: fair and unfair. Fairness refers to the acquisition of locks. If a lock is fair, the lock acquisition order should conform to the absolute time order on the request, which meets the FIFO. ReentrantLock (); ReentrantLock ();
public ReentrantLock() {
sync = new NonfairSync();
}
Copy the codeIf true, the lock is fair; if false, the lock is not fair.
public ReentrantLock(boolean fair) {
sync = fair ? new FairSync() : new NonfairSync();
}
Copy the codeIn the above unfair lock acquisition (nonfairTryAcquire method), it simply obtains the current state for some logic processing, and does not consider the current synchronization queue thread waiting situation. Let’s take a look at the processing logic of fair lock, the core method is:
protected final boolean tryAcquire(int acquires) {
final Thread current = Thread.currentThread();
int c = getState();
if (c == 0) {
if(! hasQueuedPredecessors() && compareAndSetState(0, acquires)) {setExclusiveOwnerThread(current);
return true; }}else if (current == getExclusiveOwnerThread()) {
int nextc = c + acquires;
if (nextc < 0)
throw new Error("Maximum lock count exceeded");
setState(nextc);
return true;
}
return false; }}Copy the codeThe logic of this code is essentially identical to that of nonfairTryAcquire, the only difference being the addition of the Logical judgment used to determine whether the current node has a precursor node in the synchronous queue known by the name of the method. If there is a precursor node, the thread requests resources earlier than the current thread. According to fairness, the current thread fails to request resources. If the current node does not have a precursor node, then it is necessary to make the following logical judgment. A fair lock obtains the lock from the first node in the synchronization queue every time, whereas a non-fair lock does not necessarily mean that the thread that has just released the lock can acquire the lock again.
Fair locks VS unfair locks
A fair lock is the first node in the synchronization queue to ensure the absolute sequence of resource requests. However, if a non-fair lock is used, the thread that has just released the lock may continue to acquire the lock next time, and other threads may never obtain the lock, resulting in “hunger”.
Fair lock To ensure the absolute sequence in time, frequent context switching is required. Non-fair lock reduces context switching and performance overhead. Therefore, by default, ReentrantLock selects an unfair lock to reduce some context switches and ensure higher throughput.
ReentrantReadWriteLock
To address thread-safety issues in concurrent scenarios, exclusive locks are used almost frequently, usually using the Java keyword synchronized or ReentrantLock, which implements the Lock interface in the ConCurrents package. They are exclusive locks, meaning that only one thread can acquire the lock at a time. In some business scenarios, most of the data is read, but very little data is written. If the data is read only, data correctness is not affected (dirty reads occur), and if the exclusive lock is still used in such a business scenario, this is obviously a performance bottleneck. Java also provides another ReentrantReadWriteLock(read/write Lock) that implements the Lock interface. Read-write allows access by multiple reader threads at the same time, but all reader threads and other writer threads are blocked when the writer thread accesses them. The mutual exclusion of WirteLock and ReadLock can be analyzed in terms of WriteLock to WriteLock, WriteLock to ReadLock, and ReadLock to ReadLock. Here is a summary:
Fair selection: support unfair (default) and fair lock acquisition methods, throughput or unfair is better than fair; Reentrant: Support reentrant, read lock can be acquired again, write lock can be acquired again, but also can be acquired read lock; Lock degradation: A write lock can be degraded to a read lock by following the sequence of acquiring a write lock, acquiring a read lock, and releasing the write lock
To thoroughly understand read/write locks, you must understand the following questions: 1. How do read/write locks record read/write states separately? 2. How are write locks acquired and released? 3. How are read locks acquired and released? Let’s take a look at read-write locks with these three questions in mind.
Write lock,
Write lock acquisition
The implementation of the synchronization component aggregates the synchronizer (AQS) and implements the synchronization semantics of the synchronization component by overwriting the methods in the synchronizer (AQS). Therefore, write locks are still implemented this way. A write lock cannot be acquired by multiple threads at the same time. It is obvious that the write lock is an exclusive lock, and the synchronization semantics of the write lock are implemented by overwriting the tryAcquire method in AQS. The source code for:
protected final boolean tryAcquire(int acquires) {
/*
* Walkthrough:
* 1. If read count nonzero or write count nonzero
* and owner is a different thread, fail.
* 2. If count would saturate, fail. (This can only
* happen if count is already nonzero.)
* 3. Otherwise, this thread is eligible for lock if
* it is either a reentrant acquire or
* queue policy allows it. If so, update state
* and setowner. */ Thread current = Thread.currentThread(); Int c = getState(); Int w = exclusiveCount(c);if(c ! = 0) { // (Note:ifc ! = 0 and w == 0thenshared count ! The current thread fails to acquire the write lock if the read lock has been acquired by the reader thread or the current thread is not the one that has acquired the write lockif(w == 0 || current ! = getExclusiveOwnerThread())return false;
if (w + exclusiveCount(acquires) > MAX_COUNT)
throw new Error("Maximum lock count exceeded"); // Reentrant acquire // 3.2 The current thread acquires the write locksetState(c + acquires);
return true; } // 3.3 The write lock is not acquired by any thread. The current thread can acquire the write lockif(writerShouldBlock() || ! compareAndSetState(c, c + acquires))return false;
setExclusiveOwnerThread(current);
return true;
}
Copy the codeExclusiveCount (c) : exclusiveCount(c)
static int exclusiveCount(int c) {
return c & EXCLUSIVE_MASK;
}
Copy the codeStatic final int EXCLUSIVE_MASK = (1 << SHARED_SHIFT) – 1; EXCLUSIVE_MASK is 1 shift 16 bits to the left and then decrease by 1 to 0x0000FFFF. The exclusiveCount method matches the synchronization state (state is int) with 0x0000FFFF, that is, the lower 16 bits of the synchronization state. So what does the lower 16 mean? The ‘exclusiveCount’ method is used to specify the number of write locks acquired. The lower 16 bits of the synchronization state represent the number of write locks acquired. There is another method worth noting:
static int sharedCount(int c) { return c >>> SHARED_SHIFT; }
Copy the codeInt c (int c, int c, int c, int c, int c, int c); Now remember the first question we had to understand at the beginning? The answer to the question of how read-write locks record the states of read-write locks and write locks separately is now clear. Now let’s go back to tryAcquire. The main logic of tryAcquire is as follows: When the read lock has been acquired by the reader thread or the write lock has been acquired by another writer thread, the write lock fails to be acquired. Otherwise, get success and support reentrant, increase write state.
Write lock release by overriding AQS tryRelease method
protected final boolean tryRelease(int releases) {
if(! isHeldExclusively()) throw new IllegalMonitorStateException(); Int nexTC = getState() -releases; If the write status is 0, release the write lock. Boolean Free = exclusiveCount(nexTC) == 0;if (free)
setExclusiveOwnerThread(null); //3. If the value is not 0, the synchronization status is updatedsetState(nextc);
return free;
}
Copy the codeReduce write state int nexTC = getState() -releases; The reason we just subtract the write state directly from the current synchronization state is because the write state we just said is represented by the lower 16 bits of the synchronization state.
Read lock,
Read lock is not an exclusive lock, that is, the lock can be acquired by multiple reader threads at the same time, that is, a shared lock. In accordance with the introduction of AQS, we need to rewrite AQS methods tryAcquireShared and tryReleaseShared to implement the synchronization semantics of shared synchronized components. The method of obtaining read lock is as follows:
protected final int tryAcquireShared(int unused) {
/*
* Walkthrough:
* 1. If write lock held by another thread, fail.
* 2. Otherwise, this thread is eligible for
* lock wrt state, so ask if it should block
* because of queue policy. If not, try
* to grant by CASing state and updating count.
* Note that step does not check for reentrant
* acquires, which is postponed to full version
* to avoid having to check hold count in* the more typical non-reentrant case. * 3. If step 2 fails either because thread * apparently not eligible or CAS fails or count * saturated, chain to version with full retry loop. */ Thread current = Thread.currentThread(); int c = getState(); The current // thread fails to acquire the read lock if the write lock has been acquired and the current thread is not the current threadif(exclusiveCount(c) ! = 0 && getExclusiveOwnerThread() ! = current)return- 1; int r = sharedCount(c);if(! readerShouldBlock() && r < MAX_COUNT && //2. CompareAndSetState (c, c + SHARED_UNIT)) {//3. The following code is mostly new, such as the getReadHoldCount() method // that returns the number of times the current read lock was acquiredif (r == 0) {
firstReader = current;
firstReaderHoldCount = 1;
} else if (firstReader == current) {
firstReaderHoldCount++;
} else {
HoldCounter rh = cachedHoldCounter;
if(rh == null || rh.tid ! = getThreadId(current)) cachedHoldCounter = rh =readHolds.get();
else if (rh.count == 0)
readHolds.set(rh);
rh.count++;
}
return1; } //4. Handle that the spin that failed the CAS operation in step 2 is reentrantreturn fullTryAcquireShared(current);
}
Copy the codeNote that when the write lock is acquired by another thread, the read lock fails to be acquired. Otherwise, CAS is used to update the synchronization status. In addition, the reason for the current synchronization state is to add SHARED_UNIT ((1 << SHARED_SHIFT) 0x00010000) this is because the higher 16 bits of the synchronization state we mentioned above are used to indicate the number of times the read lock was acquired. The fullTryAcquireShared method is used if the CAS fails or if the thread that has already acquired the read lock acquires the read lock again.
Read lock release read lock release is implemented through the tryReleaseShared method.
protected final boolean tryReleaseShared(int unused) { Thread current = Thread.currentThread(); // To implement new functions such as getReadHoldCountif (firstReader == current) {
// assert firstReaderHoldCount > 0;
if (firstReaderHoldCount == 1)
firstReader = null;
else
firstReaderHoldCount--;
} else {
HoldCounter rh = cachedHoldCounter;
if(rh == null || rh.tid ! = getThreadId(current)) rh =readHolds.get();
int count = rh.count;
if (count <= 1) {
readHolds.remove();
if (count <= 0)
throw unmatchedUnlockException();
}
--rh.count;
}
for(;;) { int c = getState(); Int nexTC = c-shared_unit; int nextc = c-shared_unit;if (compareAndSetState(c, nextc))
// Releasing the read lock has no effect on readers,
// but it may allow waiting writers to proceed if
// both read and write locks are now free.
returnnextc == 0; }}Copy the codeLock down
A write lock can be degraded to a read lock, and a write lock cannot be upgraded. The following example is from the ReentrantWriteReadLock source code:
void processCachedData() {
rwl.readLock().lock();
if(! cacheValid) { // Must releaseread lock before acquiring write lock
rwl.readLock().unlock();
rwl.writeLock().lock();
try {
// Recheck state because another thread might have
// acquired write lock and changed state before we did.
if(! cacheValid) { data = ... cacheValid =true;
}
// Downgrade by acquiring read lock before releasing write lock
rwl.readLock().lock();
} finally {
rwl.writeLock().unlock(); // Unlock write, still hold read} } try { use(data); } finally { rwl.readLock().unlock(); }}}Copy the codeReference articles and books:
“The Art of Java Concurrent Programming” “Practical Java high Concurrent Programming” blog.csdn.net/qq_34337272… www.jianshu.com/p/a5f99f253… www.jianshu.com/p/506c1e38a…
Please stamp: making the original https://github.com/wangzhiwubigdata/God-Of-BigData pay close attention to the public, push, interview, download resources, focus on more big data technology ~ data into the way of god ~ update is expected to 500 + articles, has been updated 50 + ~Copy the code