preface
The last article outlined the evolution, advantages and disadvantages of microservices. For complex business projects, microservices are a more appropriate solution; For us developers, there are good solutions to follow, but not blindly follow the popular technology, the purpose is to solve the problem. Here is the Asp.NetCore landing microservice architecture technology stack summary (of course not limited to this), at the same time to develop a learning and sharing plan, and the boys to encourage together;
The body of the
The technology stack involved is classified into the following stages, and the general direction of subsequent learning and sharing is the same:
For the services related to business analysis in the requirements phase, testing phase and the final application phase, this series will not be involved for the time being, but mainly for the summary and learning of development technology, code management, application deployment, operation and maintenance management technology. In many large companies, there may be a clear division of responsibility for each of these phases, but DevOps is inevitable, so it’s important to understand and learn. If it is a small and medium-sized company, it may be your development, deployment, operation and maintenance or you, technology is not pressure, to do, not to learn.
Note: It is not the use of the technology mentioned below in the project that is microservice, and microservice refers to the micro of the business, the technology is only to implement it;
The development phase
For the implementation, there are many components or frameworks involved in the development phase, so it takes more time to learn and practice, as follows:
Certification center
When the number of divided services increases, the authentication and authorization of a single service becomes more redundant. It is more desirable to have a unified authentication site. Each service is authenticated and authorized by the authentication center site. We can implement OAuth 2.0 and OIDC from zero to provide authorization and authentication functions, wheels must be made good, take to use more good, IdentityServer 4 authorization and authentication are very good implementation, so that developers have more time to focus on business development.
Service discovery
When there are not many services, it is possible to configure the API address manually, and the implementation call can accept the configuration complexity. If you are adding server extensions to a service, you have to manually configure the API address, and the person doing this will be swearing. If you use Consul to do service discovery, automatically discover each service, but also can perform a health check, filter out unavailable services in time, enhance high availability, appear more intelligent; Reducing the complexity of manual configuration also improves high availability of services.
Gateway (Gateway)
The gateway is used for route conversion, unified entry, and Intranet isolation. Of course, the gateway can also perform related functions such as service circuit breaker, traffic limiting, retry, and caching. What does the function mean and how to achieve it? We will talk about it when we do learning and sharing. In NetCore, Ocelot and Kong are common gateways;
- Route translation: According to the configuration rules, different service addresses are translated to the corresponding micro-service address, so that the service request is processed by the corresponding service API.
- Unified entry: For the UI, only one entry is concerned, that is, the gateway address, and there is no need to directly deal with each microservice, reducing the complexity of request access;
- Intranet isolation: The gateway address is provided externally. Internal services communicate through the Intranet, enhancing site security.
- Other functions will be mentioned in the following practice;
Inter-service communication
Although it is divided into small services, it is still a system. For some services of business dependency, data will be aggregated or returned to the UI layer through communication between services. The common technical means is to use Restful Api interface or gRPC for data interaction, and then data business processing.
Service governance
Each service is a program that will likely Bug and abnormal network communication, server goes down, and so on and so forth have led to the service is not available, we usually ideal must hope to other services is not affected by abnormal service, so you need to service management, such as failure retry, service fusing, degradation, etc., so as to improve service availability, Avoid the phenomenon that a single abnormal service leads to an avalanche of the entire system. The Polly library is used in.NetCore.
NoSql- A non-relational database
Generally high concurrency scenarios require caching and data sharing to achieve high availability, and currently Redis is a good choice. For some document-based data, MongoDB has more advantages in storage. When there is a large amount of data, there will be some columns of data stored. For the search implementation, the ElasticSearch storage implementation is more context-appropriate;
The message queue
Three benefits of message queues: asynchronous processing, decoupling, and peaking; In micro service system in the business process, usually encountered some complex business, it takes a long time, this gives the user experience is not friendly, we can be involved in related business through distributed to different service handles the message queue, and timely response to the user, the business background asynchronous processing, experience the feeling is different; There may be a small partner will say, direct multithreading not to get, if so, that business may be coupled together, later maintenance is a big problem; For some high concurrency systems, it is estimated that the server can carry the request at ordinary times, but there is a certain period of peak access, if the request is called to the background service database, the database may not resist, such a short period of peak situation, you can use the message queue to peak, to avoid breaking the system at peak time; The most commonly used message queues are Kafka, RocketMQ and RabbitMQ.
CAP
Microservices are distributed, and since they are distributed, there is no way to avoid distributed transactions. Finally, the problem of data consistency needs to be solved. This is an old problem for distribution, and there are solutions to it. Net, there is a big god to encapsulate CAP, and open source, with the message queue to achieve a good distributed transaction control;
Apollo
Just think, if each service has its own set of configuration files, the deployment and operations is very headache, but for some common configuration data will be repeated in each service configuration, if a set of unified configuration center is feeling very good, all service configuration data through a point for maintenance and use, No matter in development maintenance, deployment or operation and maintenance aspects, bring convenience; You can implement it yourself, you can use third parties, Apollo is relatively popular right now, and there are some that use Consul directly as the configuration center; Of course there is also the configuration center that comes with the K8S.
Background tasks
Since the microservice project is used, the data volume should not be small, and usually do some report analysis, data synchronization and other operations, and such time-consuming operations do not want to be performed in the peak business period, need to be timed in the idle time operation, for this scenario, background scheduled tasks are useful. Of course, not only this scenario, there are some business data replay or repair, such as an order in the operation asynchronous processing failed, can be in the background task check process, processing again; In.net used relatively more are Quartz.Net and Hangfire.
Code management
Code should not need to say more, friends should be familiar with the way, simple draw a picture, as follows:
Code specification
Regardless of the architecture, code specifications are always strictly required by developers, and good coding specifications must be in place during the development process. Although the requirements are different from each company, the ultimate goal is the same: standardization, so that later maintenance does not spend a lot of time studying why the code was written the way it was. Periodic Code reviews are essential for specification.
Code versioning
For code version management tools, use most should be to SVN and git, which for the management of the branch is very important, such as an interim fix bugs to do online, do regular development, functional development how emergency treatment, good programming code branch, won’t make version chaos, leading to incomplete or abnormal function; Don’t underestimate this. There are many instances where production functions fail due to version branching issues. There is not a separate article on this, but it will be covered in the integration deployment section. Because continuous integration cannot be done without code management;
The deployment of
When it comes to deployment, some people may say, “Isn’t this the work of operation and maintenance?” or “Shouldn’t there be a special person to do it?” Yes, ideally, but the truth is, you have to be in charge of not only development, but deployment, and for teams with clear responsibilities, at least you have to be, or there will be a lot of awkward docking.
The operating system
Now deployment is more recommended on Linux, such as Redis, ES, Nginx and so on are in Linux to play a better performance, but for Linux estimated some friends are not familiar, or even just heard, haven’t operate, don’t say much, about the development and deployment related daily operations when we talk about it, Advanced operations can be studied at some time. Of course, Windows operation can also be mentioned at that time, after all, now Windows server is not said not to use.
Continuous Integration (CI/CD)
The old-fashioned manual publishing and deployment is too much for microservices. With so many services and repeated operations, it’s too much for any one person to do. If you add a few more iterations, that person will take care of the release and not do anything else. Think about it, it would be nice to automate all of this, and Jenkins could do it perfectly with code management software, such as pull code -> build -> deploy. Code management software can build it itself, such as Gogs, or use GitLab, Github, etc. By listening for code submissions, you can do it automatically. Think about it.
The container is changed
Development and operation and maintenance fight, one said in my line, one said that I can not, don’t say so much, fight first say; Ha ha ha, in order not to allow this kind of thing to happen, container appears very diaosi, development release package generate mirror, transport vera down directly run, what are the same, and say it; In fact, the main purpose is to improve work efficiency. Docker is popular nowadays, but it seems difficult to say whether the abandonment of K8S can make it decline.
The container arrangement
When the container cluster expands, you need something to manage, otherwise the expansion will be super troublesome, and K8S is very popular, for container cluster management better automatic scaling, automatic deployment, but also with probe to achieve self-repair;
operations
Function development finished, the code has been uploaded, the site has been deployed, the user began to use it, the follow-up is very easy; No, no, no, it didn’t start, should be few friends clap breast said, that’s ok, function is no problem for me to do, and absolutely no Bug, good, hypothesis development no problem first, power outages, downtime, broken network to do, the physical problems can’t avoid it, whether it’s doing different live, or have other solution, at least to get; If it is a business problem, troubleshooting is more dependent on logging. For microservices, a call chain tracing will provide a good aid. Let’s see what technical tools are needed first:
Log analysis
Before the single architecture, on the server, copy log down analysis is done, but for micro services, this seems not to be desirable, copy log is not only troublesome and time-consuming. If the Exceptionless logs are collected together uniformly, would it be a little better? In addition, the query analysis by ElasticSearch and Kibana would speed up the analysis of the problem?
Link to track
Data interaction between microservices cannot be avoided. A visual call link and monitoring makes it more intuitive. It can clearly see the service point passed by each interface call, which also provides great help for locating problems. SkyWalking and Butterfly are used more often.
Distributed monitoring system
In the case of microservices, as long as the release, do not know what the situation, so only after the problem occurs, can go to the investigation; Is there a monitoring system that monitors the environment in which the system and services are running, and can alert and notify personnel when they are in danger? Prometheus and Grafana can provide data friendly display and configuration to monitor the environment in which the services are running.
conclusion
For microservices, as mentioned in the beginning, either the above technologies are used or the microservice architecture is used. The division of services is more based on business, and technology only helps it to be implemented.
The above technology stack summary for each stage does not cover all, but is relatively popular at the moment, and many friends use it, friends can share any good technology.
This is a long line, but certainly will insist on learning and sharing. During the Redis series after sharing, will be interspersed with the database optimization series of articles.
A handsome guy who was made ugly by the program, follow “Code Variety Circle “, learn with me ~