Operation and maintenance is about omnipotence, omnipotence

[TOC]

Hi, I’m Stanley “Stanley” and today we’re going to talk about what pitfalls you should avoid when containerizing microservices.

In the previous article, we talked about cost control in containerization.

The first job sensitivity in operations is cost. There is no objection.

Once, an architect recommended the boss to containerize for a reason that containerization could save the company’s capital cost in a wide range. I also agreed with it for a while, but later I found that everything has a premise.

There is no denying that microservices are inherently containerized, but only if they are properly unbundled!

Too rough microservice separation will at most affect the iteration of the development working version and increase the difficulty of version control, but too detailed microservice separation will increase the cost after containerization.

To be safe, Java processes start with a separate memory allocation declaration

-javaagent:/usr/local/skywalking_agent/skywalking-agent.jar -Dskywalking_config=/usr/local/skywalking_agent/config/app.config --eureka.client.serviceUrl.defaultZone=http://eureka.example.com/eureka/ -XX:PermSize=128m -XX:MaxPermSize=128m
Copy the code

The same goes for k8S, which officially recommends a quota configuration for all pods.

        resources:
          requests:
            cpu: 2
            memory: 2048Mi
          limits:
            cpu: 2
            memory: 2Gi
Copy the code

The above method is hard setting for resources, that is, fixed allocation of 2c2G resources. If you think this method is too wasteful of resources, K8S also provides the following human Settings.

limits:
- default:
    cpu: 800m
  defaultRequest:
    cpu: 800m
  max:
    cpu: 800m
  min:
    cpu: 200m
  type: Container
Copy the code

Soft resource limits cache wasted resources, but they do not substantially address wasted resources because no one knows when POD usage will peak, and even if they do, they need to always be reserved for them, which is safer than hard resource limits.

So, the question comes back to how to microserve the cutting principle.

First, micro services common pit

Monomer applications run after a period of time, as the growth of the business, the system performance and concurrency requirements more and more high, this time is faced with micro service refactoring options, but in front of the reconstruction, we must weigh repeatedly and prepare for the necessary infrastructure to cope with the new architecture faces new problems, rather than think micro service began their micro now head a hot fire service refactoring.

In combination with the actual project, the possible pits of microservices are reviewed, mainly including the following points:

  • Too much emphasis on the “micro”, resulting in excessive service separation, too many services, resulting in complex relationships between services, system complexity, high labor and maintenance costs;
  • The original single interprocess service invocation becomes remote interprocess service invocation, and the invocation mode changes from in-memory invocation to network invocation. If the invocation link is too long, it will lead to performance degradation, longer response time, and more difficult problem location.
  • Poor infrastructure, lack of necessary monitoring and governance of services, resulting in a mess of service management, what was supposed to be lightweight microservices becoming a mess, and better automated testing and deployment support, otherwise fast delivery cannot be achieved with dependencies between services.

We will introduce it in several aspects:

  • Prerequisites for Service Splitting
  • Service Split Timing
  • Service split method
  • Service resolution specification

2. Premise of service split

  • The CICD platform of the company is relatively complete

After service separation, the workload brought by synchronization, such as verification, testing and on-line, is increasing. The process of micro service is actually a process of human resources for quality. If the supporting tools are not mature, the complexity of synchronization can outweigh the convenience.

  • The RESTFUL API interface is separated from the front and back ends

The new generation of development frameworks basically have a good separation of the front and back ends. This has the advantage of being transparent to the front end, no matter how the back end changes. In addition, gray scale publishing, routing distribution and traffic segmentation can be realized in the process of splitting, so as to ensure smooth splitting

  • Database table design

There is not a large number of complex joint queries between functional modules, and excellent library table design should be able to achieve the vast majority of functional queries are only O(N) queries of K/V. Otherwise, microservices can be very difficult to break down, just like code refactoring

  • Stateless,

The new generation of development frameworks and popular open source tools are emphasizing features that are decentralized and stateless. Statelessness is also one of the core ideas of K8S.

3. Timing of service separation

New businesses typically use springcloud buckets, which are microservitized to some extent or fully.

Service of the old business can reform the split is not a one-off reengineering, is usually driven by pain points, is a business really met rapid iteration and high concurrency problem, if you don’t break up, will bring the influence to the development of business, only this time, the service of the split is to determine the income, increase the operational cost is worth it.

For example, one of the following situations:

  • Commit code frequently collides
  • Unable to quickly version iteration, small functions must be accumulated to the big version before the launch, each time the launch of various problems, do not excrete three layers of skin online unsuccessful
  • Services have high concurrency requirements
  • All fuses are off if-else

In this case, consider service split

4. Service splitting method

AKF Scale-up Cube (The Art of Scalability) is an abstract application of three dimensions to The Scalability of a company called AKF. Theoretically, according to these three expansion modes, a single system can be extended indefinitely.

  • X-axis: refers to horizontal replication, which is easy to understand. It means that a single system runs several instances to make a cluster and load balancing mode.

  • Z-axis: it is based on similar data partition. For example, an Internet taxi-booking application suddenly fails, the number of users increases rapidly, and the cluster mode cannot hold up. Then, data partition will be carried out according to the region requested by the user, and more clusters will be built in Beijing, Shanghai, Sichuan, etc.

Y-axis: This is what we call the microservices unbundling model, which is based on different services.

Scenario description: For example, jingdong Mall, a cluster can not support, divided into multiple clusters, and then the user surge is not enough, after analysis found that 6.18 page traffic is very large, the mall page traffic function to do a split, each split out of the function, independent maintenance, each can be expanded according to demand again.

What not to do:

The internal application system of our company has less than 100 users, but the system is divided into more than 30 modules, resulting in a large waste of resources in containerization.

5. Service separation specification

The service separation specification is not a strength of operations, or operations does not have much opportunity and say to participate in the actual development of development.

In actual work, microservices have a great impact on operation and maintenance work. For example, the workload increases sharply and the resource cost is consumed more, but the service quality is not actually improved, so the priority is given

  • Whether o&M tools are mature and sound
  • Is the CICD process mature and reliable

If none of them, then consider the problem of micro-service split, here is only the introduction, it is recommended to further study and then improve and optimize together with the development students.

  • Service split specification 1: service split up to three layers, two calls
  • Service separation specification 2: only one-way invocation, no circular invocation
  • Specification 3 for service separation: change serial calls to parallel calls, or asynchronization
  • Specification 4 for service separation: Interfaces should be idempotent
  • Service split specification 5: Interface data definition is strictly prohibited to be embedded or passed through
  • Specification 6 for service separation: Normalized project names

Microservices specification reference: dockone. IO /article/824…

For more exciting content, please visit me at my official account: Linux178. 200+ high-quality original articles waiting for you