Click “like” to see again, form a habit, wechat search [Three prince Aobing] to pay attention to the Internet tools of survival.

This article has been included in GitHub github.com/JavaFamily, there are a line of large factory interview complete test sites, information and my series of articles.

preface

Lock I don’t think I need to say too much, but everyone knows what’s going on, right?

In a multi-threaded environment, data may be inconsistent or polluted due to context switching. We need to ensure data security, so we think of locking.

The so-called locking mechanism is that when one thread accesses a certain data of the class, it protects the access from other threads until the thread finishes reading the data and other threads can use it.

Remember I said that Redis needs to lock data with concurrent contention in the distributed case, you are confused, Redis is single-threaded? Why lock it?

You don’t need to lock it.


When accessing Redis from a single service, there is no need to consider thread safety because Redis is single-threaded. However, which company is still single-threaded? It must be a distributed cluster.

Look at this scenario and see if there is a problem:

You often don’t say kill seconds, get inventory judgment, that wife tells you distributed situation is going to be a problem.


In order to reduce the pressure of DB, we preheated the inventory to KV, and now the inventory of KV is 1.

  1. Service A went to Redis to check the inventory and found that it was 1, which means I can grab this product, right? Then I was going to reduce by 1, but it hasn’t been reduced yet.
  2. At the same time, service B also went to find 1, that I also grabbed ah, that I also reduce.
  3. C in the same way.
  4. When all the services are finished, you find out why it’s -2, it’s oversold, it’s over.

I will divide into three chapters to introduce three implementation methods of distributed lock (Zookeeper, Redis, MySQL) respectively, and describe their advantages and disadvantages, as well as general practice scenarios of big factories.

The body of the

A SAO li SAO qi of the interviewer what also did not take of walked in, you see, this is not your wife, you are preparing to call him, he found a serious face, dead ghost also installs serious, affirmation will give me water of.

B station search: the third prince Ao Bing

Well, let’s stop saying anything and start today’s interview.

What are the mechanisms for normal thread process synchronization?

  • Mutually exclusive: A mutually exclusive mechanism that ensures that only one thread can operate shared resources, such as synchronized and Lock, at a time.
  • Threshold: allow multithreaded serial calls to access resources
  • Event notification: Notifies users of events to ensure orderly access to shared resources
  • Semaphore: Multiple tasks are accessed simultaneously, limiting the number of tasks at the same time, such as starting gun CDL, Semaphore, etc

Do you know what distributed locks are?

Distributed lock is mainly implemented by Zookeeper (hereinafter referred to as ZK), Redis and MySQL.

So tell me about ZK first, can you tell me about its common use scenarios?

Its main application scenarios are as follows:

  • Service registration and Subscription (shared nodes)
  • Distributed notification (listening on zNodes)
  • Service Naming (ZNode feature)
  • Data Subscription, Publishing (Watcher)
  • Distributed locks (temporary nodes)

Zk is what?

It is a database, file storage system, and has a listening notification mechanism (observer mode)

Save the file system. What did he save?

node

There are four types of ZK nodes

  • Persistent node (ZK disconnection node still exists)

  • Persistent sequential numbering of directory nodes

  • Temporary directory node (deleted after client disconnects)

  • Temporary directory Number Directory node

Node names are unique.

How is a node created?

What the hell am I asking? But I only saw distributed locks in the interview, I have to think about it!!

Fortunately, I set up a zK cluster on my own server, so I just want to remind you of a wave.

Create /test laogong // Creates a permanent nodeCopy the code

What about temporary nodes?

Create-e /test laogong // Creates a temporary nodeCopy the code

The temporary node is created successfully. If I disconnect this link, the node will disappear naturally. This is one of my ZK management tools.


What about creating sequential nodes?

Create-s /test // Creates a sequential nodeCopy the code

What about temporary sequential nodes?

I think all the smart guys will answer

Create-e -s /test // Creates a temporary sequence nodeCopy the code

After I quit, I reconnected and found that all the temporary nodes I had just created were gone.



The beginning of the demonstration so much, I just want to give you see zK about an operation process and data structure, the middle involved in the construction and other skills I will not say, we focus on talking about his implementation in distributed lock.

Zk is based on nodes to achieve a variety of distributed locks.

Taking the initial scenario as an example, how can ZK ensure thread safety in distributed situations? How does he control concurrent competition?

To simulate concurrency contention, I’ve written some pseudo-code that you can look at


I defined an inventory value of 1 and used a CountDownLatch starter gun to reduce inventory when all 10 threads were ready.

Is it like 10 machines going together to get the inventory and then deducting the inventory?

When all the machines were taken together and all the machines were found to be 1, everyone thought that they had grabbed the machine and they reduced the value by 1. However, when they went to set value after all the machines were executed, they found that it was already oversold. I will print it out for everyone to have a look.

image-20200406143556640

Right? This is not a problem of overselling one or two, there are seven overselling, the code clearly judged that the inventory is greater than 0 before the reduction, why I explained at the beginning.

So how do we solve this problem?

Sync and lock can only keep your current machine thread safe, so distributed access is still a problem.

The zK nodes mentioned above can solve this problem.

Zk node has a unique feature, is that we have created this node, you create ZK is an error, then we will use his uniqueness to achieve it.



How do you do that?

Aren’t there 10 threads up there?

We create all of them, the first one that succeeds returns true and it can proceed with the destocking operation, and all subsequent node accesses will fail, destocking will fail, and we will queue them up.

So how do you release the lock?

Delete the node, delete the node and then notify the other people to come to lock, and so on.

Let’s implement the zK lock scenario.

image-20200406151627845

Right? Well, only the first thread succeeds in deducting, and all the others fail.

But you found the problem did not, you added the lock, you have to release ah, you do not release the back of the error will not retry.

Lock is in finally, unLock, and now we delete the node in finally.

We know that creating a node is enough, but you have to implement a blocking effect.

An endless loop of recursion that keeps trying until it succeeds, a disguised blocking effect.

How do you know the front brother deleted the node?

Listen for node deletion events



But do you see the problem with that?

Yes, there will be deadlocks.

The first lock was successfully added. When executing the code, the machine went down, so the node could not be deleted.

Be reflective, ask yourself questions, look at the distance, look at the interviewer, pretend you don’t know anything.

Oh, I remember, just create a temporary node. Once the client is disconnected, other nodes can listen for changes in the node.

Well, not bad. Do you see any other problems?

Sounds like that’s not a good listening mechanism either.

What’s wrong with that?

As you can see, listening, where all the services are listening for one node, and the release of the node is also notified to all the servers, what if it’s 900 servers?

This is a big challenge for the server, a release message, like a sheepdog in the sheep, everyone scattered, could kill the machine at any time, will take up service resources, network bandwidth and so on.

This is herding.


So how do we solve this problem?

Continue to pretend to think, ah ah ah, it is difficult, my brain…

Will you stop fucking pretending?

All right, temporary sequential nodes, that solves this problem.

How to achieve you don’t look down, first think about yourself.

As mentioned before, it is very problematic to listen on all one node, so we will listen on our previous node, because it is sequential, it is easy to find their own before and after.


The difference between listening on a permanent node and listening on a permanent node is that each node only listens on its previous node and releases it one by one, so there is no herding effect.


Above all the code I will be open to my https://github.com/AobingJava/Thanos actually the minor defects above, we can go to pull off, change to submit pr, I will see right through.

That’s great, but can you talk about some of the weaknesses of ZK’s distributed locking practices?

Zk may not perform as well as caching services.

Because every time in the process of lock creation and release, the dynamic creation and destruction of instantaneous nodes to achieve lock function.

The creation and deletion of nodes in ZK can only be performed by the Leader server, which then synchronizes data to all Follower machines. (HERE involves the knowledge of ZK cluster, I will not expand, later ZK chapter with you in detail)

Still have?

There are concurrency issues with Zookeeper, but they are not common.

Due to network jitter, the session connection of the ZK cluster is broken by the client, so ZK thinks that the client is down, and will delete the temporary node, at this time other clients can obtain the distributed lock.

Concurrency issues can arise, which is uncommon because the ZK cluster has a retry mechanism that retries once the ZK cluster cannot detect the client’s heartbeat. The Co-curator client supports multiple retry strategies.

The temporary node will be deleted if it does not work after multiple retries.

It is also important to choose an appropriate retry strategy that strikes a balance between lock granularity and concurrency.

Is there a better implementation?

Distributed lock based on Redis

Can YOU talk to me for a second?

I looked at the hand of the table, it is not early today, you all asked, I how to water a few articles?

Well, it’s late. Why don’t you go home and do your chores?

I ？？？？

=

conclusion

Zk solves the deadlock problem by using temporary nodes. If the client suddenly hangs after acquiring the lock (the Session connection is disconnected), the temporary node is automatically deleted and other clients automatically acquire the lock.

Zk also realizes the principle of blocking through the mechanism of queue monitoring of nodes. In fact, it is a recursive process that waits indefinitely for the release of the smallest node.

I didn’t implement reentrant lock above, but it is easy to implement, can be with thread information, or machine information such as a unique identifier, determine when to obtain.

Zk’s cluster is also highly available, with only half or more of them being able to provide services externally.

Redis and database distributed locks will be written this week, you wait.

I am Aobing, a tool to survive on the Internet.

The best relationship is mutual achievement, the husband’s ** “three even” ** is the biggest power of the creation of Propylene, we will see you next time!

Note: if this blog has any mistake and suggestion, welcome the husband to leave a message, husband you quickly say a word!

[Information] [interview] [resume] I prepared the interview materials and resume template, this article GitHub github.com/JavaFamily has been included, there are big factory interview complete test site, welcome Star.


The more you know, the more you don’t know