preface
The company has a Web management system, which is deployed using Tomcat. Because it is the background management system, all the web pages need to login authorization after the corresponding operation.
In the beginning, this system was not used by many people. In order to save resources, this system was only deployed on a single machine. As more and more people used it, it became too much for a single machine, so I decided to deploy another machine.
At this time, the back-end system has two services, so we use Nginx as the reverse proxy, the overall architecture diagram is as follows:
This architecture diagram should be familiar to all of you, as it should be deployed in mainstream Web systems today.
After some debugging, the system was deployed to production in the dead of night. I thought there was no matter, very stable to test the little sister began to test.
There’s something terribly wrong with this test! Test little sister feedback, after logging in, not for a while and need to log in, operation is several times.
I checked, system application, configuration, everything is ok, so what’s wrong with it?
This time the group leader just ready to go off work, see we have a problem here, so came to have a look. A brief overview of the basic situation, soon found the cause of the problem, and then on the Nginx side to modify the configuration, restart to solve the problem.
After the first point praise, form a habit ~ pay attention to the public number “procedure”, come quickly!!
Distributed Consistency Session
After solving the problem, the group leader sat down and explained the cause: distributed consistency Session.
Originally, after login, user login information will be put into Session. Users will first verify whether the Session has user information every time they operate. If not, users will be forced to log in first.
In the original architecture, we only had one application system, and all operations were done on one Tomcat, which of course was not a problem.
But now we have two systems deployed, and since Nginx uses the default load balancing strategy (polling), requests are distributed to back-end applications one by one in chronological order.
So when we first log in to Tomcat1, the user information is stored in Tomcat1’s Session. After a while, Nginx sends the request to Tomcat2, which has no user information in its Session, so it needs to log in again.
In addition, since our system adopts single sign-on (SSO), the login information of Tomcat1 will be invalid after Tomcat2 logs in. Therefore, when Nginx sends traffic to Tomcat1, the login information of the Session user is invalid and the user needs to log in again.
Knowing the problem, of course, I want to know the solution, so the group leader taught four solutions to distributed consistency Session, and the little black brother sorted them out for us:
The following little black brother will explain the solution of distributed consistent Session in the form of dialogue with the group leader.
Session replication
The group leader:
If user information exists in Tomcat1 Session, but not in Tomcat2.
If we copy Tomcat1’s Session to Tomcat2, then Nginx forwards the request to Tomcat2. Since Tomcat2 has a Session, there is no need to log in again.
The architecture diagram is as follows:
Tomcat Session replication configuration, there are more examples online, here xiaohei will not post, interested students can search for it.
Black:
Yeah, that’s a good way to go. Tomcat supports this approach, we just need to change the Tomcat configuration, we don’t need to change the application code.
The group leader:
That’s right, but there are still many disadvantages to this approach.
First, Session replication and transmission occupy Intranet bandwidth.
Second, with only two machines in our example, the replication performance is ok. But let’s say we have N machines, and we have to copy to n-1 machines for every replication. If there are many machines, network storms can occur, and replication performance can degrade exponentially.
Third, Tomcat needs to store all Session data. Sessions in this scenario are stored in memory, which is easily limited by the total memory of the machine. We can’t scale horizontally by adding machines. What we can do is increase machine memory. But the bigger the memory, the price is really expensive!!
This is not recommended.
Session front-end storage
Black:
Well, that’s a bit of a stretch
Ah, yes! Our Session actually stores the user’s information, so I don’t save it in Tomcat Session now. I take the information out and save it in the Cookie of the browser.
In this way, each user’s browser stores its own Cookie information and the server does not need to store it, which solves the defect of the Session replication scheme.
Then each time the user requests to send the Cookie to me, I judge the user information in the Cookie is not good.
The architecture diagram is as follows:
Group leader, admire a look at me:
Yes, your plan does work.
However, if you use this scheme, first you need to figure out the encryption scheme.
User information but our sensitive data, can not let others easily steal or tamper with the data.
In addition, this scheme carries cookies for each request, which will occupy the bandwidth of the extranet. If cookies are too large, it will increase the overhead of the network.
In addition, the size of the data we store is subject to Cookie limits.
So it’s still not very common, but it’s an idea.
I recommend the following two options.
Sticky Sessions
The group leader:
As you can see, I just made a few changes to the configuration of Nginx, and the problem was resolved.
This is because I changed the default load balancing policy of Nginx to use IP Hash.
Nginx hashes the requester’s IP address and then dispatches it to a machine, ensuring that requests from the same IP address land on the same Tomcat.
The architecture diagram is as follows:
Nginx can also use 7-layer load balancing, which uses Http service attributes to Hash, such as userId,loginId, and so on.
The architecture diagram is as follows:
Black:
This seems to be a simple solution, we just need to change the Nginx configuration, not the application configuration.
As long as the source IP of the request is sufficiently random, the traffic on the two applications after the IP HASH will be sufficiently random.
In addition, if the two machines can not support later, we can scale horizontally and add more machines, just modify the Nginx configuration.
The group leader:
You are right on all those points!
But have you ever thought that, in the case of our company, everyone’s export IP is the same. Then all of our company’s requests only go to one machine, which means we’re a single point again.
In addition, if Tomcat restarts, the Session will be lost because it is stored in the memory, which will cause users to log in again.
Finally, if we temporarily add machines, Nginx will recalculate Hash distribution requests after rebooting after modifying the Nginx configuration.
In this case, some users will reroute to a new machine and need to log in again because there is no Session.
However, Tomcat does not restart or add machines many times, so this is not a problem, the user experience is a little bit worse.
That’s what we’re going to use for today’s solution to the problem.
But we’re going to do it this way.
Back-end centralized storage
The group leader:
In the above methods, we store the Session in the application memory, and the Session will be lost as soon as the application machine restarts.
To solve this problem, we save the Session separately to Redis or MySQL.
However, due to the expiration feature of Session, there is no need for persistent storage, so I recommend Redis to save.
The structure should look like this:
With this scheme, there is no risk of Session loss, as long as Redis does not break down.
In addition, if the application can be directly extended horizontally.
If the request volume of the later application is too large for one Redis to carry, we can actually do cluster expansion and do routing according to the cache Key.
Black:
Yeah, yeah, that’s a good way
The group leader:
Don’t get too excited. We have to pay a price for using this plan.
First of all, we need to call Redis once for each request, which increases the network overhead once.
In addition, with the introduction of Redis, we need to make changes to the corresponding code, so that the complexity is higher.
So, there are pros and cons to this scheme, and of course for our scenario, the pros outweigh the cons.
Black:
Well, it seems so.
The group leader:
All right, it’s late, problem solved, let’s go jerk off, on me!
Black:
Boss, 🐂!
The group leader patted black’s head:
This meal is not for nothing. Next week, please modify the current mode and change it into the mode of Session centralized storage.
As a tip, you can use Spring-Session.
Black:
Okay, short mouth. I’ll look into it next week.
conclusion
To sum up, when our back-end Web applications expand to multiple platforms, we will encounter the problem of distributed consistent Session. There are four mainstream solutions:
- Session replication: Synchronous replication using Web containers such as Tomcat
- Session front-end storage: Saves Session information using cookies in users’ browsers
- Session stickiness scheme: Nginx can Hash four or seven levels to ensure that all user requests are sent to the same machine
- Session back-end centralized storage solution: Redis is used to centrally store sessions, so that Web applications can be restarted or expanded without losing sessions.
Of the above four schemes, the fourth is preferred.
Of course, the fourth scheme needs a certain amount of development work, and the third scheme can be used for intermediate transition if the process has not been transformed in the early stage.
Spring-session (spring-Session) : Spring-Session (spring-Session) : Spring-Session
help
- The Architect’s Path – Session Consistent Architectural design practices
Welcome to pay attention to my public account: procedures to get daily dry goods push. If you are interested in my topics, you can also follow my blog: studyidea.cn
