Earlier we talked about two pain points of microservices: responsibility division of microservices and granularity separation of microservices. Here we continue to talk about the remaining pain points:

No one knows the whole picture of the overall architecture of the system

I don’t know if you have ever encountered this situation: every few months or six months, the big leaders will ask us to report the number of micro-services in each department, the total number of micro-services in the company, and what each micro-service is used for. Because of the large number of micro services in the enterprise, it is always a long list when reporting to the big leaders.

Then the big boss starts complaining: “Hundreds of microservices? Is the system that complicated? Who knows the whole picture of all the systems, and if there is a problem, how can we quickly locate the problem?” Heads were bowed, and one of them muttered, “I don’t even know my own department’s microservices list.”

In the past, we would have a complete picture of the entire architecture of the company first, and then it would be easy to locate the failure point if there was a problem. But since came to the use of micro services after the company, there is no such impulse, only to understand their own one mu three points on the line, if there is a problem to temporarily learn about the relevant system.

Therefore, in practice, it is very difficult to find someone who knows the whole picture of the overall architecture of the system, which is a pain point of microservices.

Two, repeated code

At my previous company, we put all the code in the same project, and if we found some code that could be reused, we pulled it out and put it in the Common package. But this kind of code design can often be problematic in microservices, so I’ll give you an example.

For example, one team implemented a logging auto-burying feature that automatically logs calls to specific methods. When other teams heard about this feature, they liked it and wanted to use it directly, so the Buried Point team happily gave Maven’s announcement. However, the first team to use the JAR immediately reported a conflicting JAR version problem, and if they upgrade the conflicting JAR, the original code is no longer usable. To save on labor costs, they asked the buried team how to achieve version compatibility.

To accommodate the Crab team’s version of the JAR, the Autoburied JAR team redesigned a version of the buried JAR, removing the use of certain apis, and finally the two teams were able to use it properly.

However, a third team using a buried JAR reported a JAR version conflict, at which point the auto-buried JAR team had to give up maintaining the common JAR from an input/output perspective and told the other teams that the code was in Git and you could fork it yourself. As a result, the code ended up in several different versions of microservices for different teams.

We took a second look and concluded that there was nothing wrong with reusing jars per se, but that we were using too many JAR versions and had to change that.

So we have unified all JAR versions of the project formally set up. The next day, urgent business demands came and everyone forgot about it. After a while, the central project was brought up, only to be swamped by urgent business needs. Then it became clear that the project could not be done because the input-output ratio was not high.

Actually micro service between repetitive code is ok, because the duplicated code everywhere, between the departments and technical center each department has its own framework/Common/Shared/arc GitLab subgroups, they can realize the inter-departmental generic code reuse.

However, it is much cheaper to maintain these little duplicates than to schedule refactoring and review JAR versions uniformly.

Third, consume more server resources

I once learned about a small company. They used a monolithic architecture with five servers deployed, but complained that the system was too coupled, their code was often interacting with each other, and urged that the architecture be moved.

Therefore, the original monolithic architecture was split into six microservices based on business modules. Considering high availability, each service needed to be deployed on at least two nodes, plus two servers for the gateway layer, resulting in a total of 15 servers. (Because one of the services was expensive, an extra node was added for insurance purposes.)

Does not change, in the process of the resolution, the business flow does not change, code logic changes little, but nine more server for no apparent reason, for it had spats, controversial point is that if this is the case, should not be only one server to deploy a service, for example, we can put the service AB deployment in 1 node, The BC service is deployed on one node, and the AC is deployed on the other node, as shown in the following figure:

But this scheme was quickly rejected by everyone, because if only one service is deployed on each server, the name of the server can be directly named by the name of the service, and it is convenient for o&M to troubleshoot problems later. But what about the name of the server if we mix microservices?

So, someone said, “How about this? Servers are cheaper anyway, so it doesn’t matter if you have more.” Everyone agreed. The company’s money is wasted by these programmers, but don’t think it’s only small companies that do it, big companies do it too.

A few days later, CTO called all the R&D staff to a formal meeting: “This quarter, our server budget is too much, the financial department did not approve, you need to find a way to reduce the number of servers, not the use of the server down.

After the meeting, everyone went back to their desks and started checking each service, which led to the following conversation.

A: “How does this service use so many servers? Is it resource-intensive?” B: “No, the company forced us to implement multi-data center deployments.” A: “Is this service important? For internal use?” B: “Yes, this is currently only used by developers.” A: “Then why do load balancing, down, only one left.” B: “Ok.” A: “How many servers are we downsizing now?” B: “……”

This is common in most companies, so it’s fair to say that microservices are a real server drain.

Distributed transactions

The pain point of distributed transactions is hell for microservices. To understand this pain point, let’s take the example of the ordering process we have experienced.

The original order process is like this: insert order — > modify inventory — > insert transaction — > insert financial receipt — > return the result to the user, let the user jump.

In a singleton architecture, we simply wrap the order flow above in a transaction, and if a process fails, we simply roll back the data and inform the user through the business code that something has gone wrong and let the user try again.

However, when moving to microservices, because these processes are stored in different services, we need to update different databases, which requires the following logic.

Does a process fail and need to roll back all data? If necessary, we need to write rollback code in each process. What if the roll fails? Do we also need to write rollback code, rollback code is a rollback?

Roll back some processes and not others? So which processes roll back and which ones don’t?

Or do not unify rollback, failed to retry? Wouldn’t that have to be asynchronous? If you make it asynchronous, will there be a time out? What if the user times out? Do I need to roll back? (Why did it roll back?)

The problem is that there are so many scenarios in which distributed services update data that we’d be crazy to try to do this logic in every scenario. The business department had complained that we were too slow to deliver, and we had to spend time on this logic, so the whole department had to be disbanded.

Therefore, for this situation, in most scenarios, we do not consider rollback and retry, just consider writing Happy Path, if an error is reported, record an exception log, and then manually handle offline, So easy!

As you know, the network jitter in the computer room is a common thing (operation and maintenance often take this as a reason), so that there are anomalies in data update every three days. For example, upstream data is updated, but downstream data is not updated. At this time, data cannot be matched, which is particularly embarrassing.

So that the business department often complains: “Gee, why can’t I find the receipt for this order? Hey, how does this transaction sheet have no transaction flow?” And then we just say, “Well, I explained that to you. Just bring up a work order and whine lest the boss won’t know?”

Distributed transactions have always been a pain point and difficulty in using microservices, so we took the pain in mind and decided to solve this problem. The solution to this problem will be explained in the following section.

Interested friends welcome to follow the wechat public number: server technology selection

Personal blog: Jiangyi. Cool