This article is based on the speech of the same name in the first Meetup of “Volcano Engine Developer Community”, mainly introduces the correlation and difference between Spring Cloud technology system and Cloud native technology system, and how to build microservice system with the help of Cloud native ability.
Author | Xia Yan, Volcano Engine senior R&D engineer
Hello, everyone. I am Xia Yan, a senior R&D engineer from Volcano Engine. My topic today is to replace Spring Cloud and use Cloud Native based service governance.
About Spring Cloud technology system
We expand the whole project background through the timeline:
- When I started (before 2010), there was probably no cloud native community, and the Java architecture was the preferred choice for enterprise development.
- In 2010, Netflix launched its Move to Cloud plan to Move most of its services to AWS.
- In 2012, Netflix launched the Open Source Software Center, similar to Apahce Maven, which offers Open Source projects that have been deposited in the cloud.
- In 2014, Martin Fowler published a well-known blog called Microservices (Martinfowler.com/articles/mi…
- High maintainability and testability;
- Loose coupling between services;
- Services can be deployed independently;
- Services are organized around business;
- Used by small teams.
(Now, both cutting-edge Internet companies and traditional IT enterprises have gradually adopted microservices. In the face of increasingly complex business pressures, only microservices architecture can keep the enterprise alive and software development iterative.)
- In 2015, the Spring community launched Spring Cloud V1.0.0, which is still widely used today, based on some of the components that Netflix deposited and Martin’s idea of microservices. Spring Cloud V1.0.0 contains fewer components, with only a few core components such as service discovery and configuration management.
Therefore, the development process of micro-service architecture is not from papers to industrialization, but from the practice of engineers to abstract characteristics, and finally form a complete ecology. To date, Spring Cloud components are well developed, including configuration, service decoupling, service discovery, fusing, routing, messaging, API gateways, tracing, CI pipelines, and testing. These make up the entire Spring Cloud ecosystem.
- Spring Cloud is a microservice system built on Java. In the process of continuous iteration between Spring and the Java community, a new force has emerged. Kubernetes was first released on June 7, 2014, with competing scheduling platforms like Docker Swarm and Mesos.
As you can see from the timeline, Kubernetes and Spring Cloud were around the same time.
At the time, some emerging customers faced a question: Which model is better for java-based business application development?
For this problem, we now prefer To use Kubernetes because Kubernetes is a language-neutral platform. Spring Cloud is a JVM architecture, but you can’t do a lot of things without the JVM, so you have to force customers to make changes along with it, which is not a great experience. Therefore, we later persuaded some teams of the company to participate in the construction of CNCF cloud native technology architecture.
Spring Cloud base capability replacement
Configuration center
Spring Cloud’s Config Server has many capabilities:
- Git as a configuration repository;
- JDBC and Redis provide a uniform layer of configuration abstraction.
But it doesn’t work very well. Some personalized requirements, such as permission management and hot loading of the configuration center, are not supported by Spring Cloud Config Server itself and need secondary development.
For Kubernetes, you can use ConfigMap or Secret to inject environment variables, files, or startup parameters into the application in a more native way, just like typing a Linux command.
We will find that Spring Cloud Config Server is more like a stand-alone software and Kubernetes ConfigMap is more like an in-software feature, which is the difference between the two.
Configuration management
Kubernetes configuration management is relatively easy, just add Environment to the final startup declaration, or load ConfigMap as a Volume.
Sometimes some colleagues ask, although Sping Cloud does not have the hot loading capability originally, but based on SpringEventBus, or even with some open source tools from third-party manufacturers, it can also achieve the so-called hot loading, can Kubernetes do it?
Kubernetes can do that, too. Environment variables are immutable, of course, but mutable properties can be mounted as files into the YMAL file of the host container application. As the ConfigMap changes, YMAL will also change at the same time. In this case, you just need to make the application can watch the configuration file changes and automatically load. Hot loading should be implemented by the application itself.
Kubernetes also has reload capabilities of its own, especially when extended to other languages. Byte internal use of Go language is more, as long as you can reload a file or remote address, the application can change its behavior.
Service discovery
The biggest difference between Spring Cloud and Kubernetes is service discovery. Most of our functions need to be seconded based on service discovery, which brings us to the choice of service discovery.
Spring Cloud’s service discovery is based on Eureka (and could later be based on Consul), providing self-reporting and client load balancing, and is an AP system.
Kubernetes is more like a traditional cloud vendor, helping users create machines/containers. The platform knows where the application is and can help direct the flow through DNS and server side load balancing. The experience is quite different.
The Spring Cloud system, if it is Eureka Client, is always embedded inside the business, because it knows where the application is at the moment of startup, and obtains the current IP address through Utils components. Kubernetes doesn’t need to be aware by the application, which is a big difference.
Accessing Kubernetes’ service discovery is also relatively simple. Create a resource for a service and define its Label. I think service discovery is a great strength of Kubernetes.
Auto Scaling & Self Healing
Auto Scaling and Self Healing are not available in Spring Cloud. In Spring Cloud, Eureka does some health checks. The logic is simple: Eureka keeps sending requests to see if the heartbeat is regularly reported. But Spring Cloud can only know if a service is healthy, not prevent access to unhealthy services. If you want to scale up or self-recover unhealthy services, you need to do a lot of extensions in Spring Cloud.
Kubernetes does this a little better. It provides readless detection itself, after which the platform helps with automatic extension and scheduling if the call fails. It is also as simple as opening a port in the application or container that can detect whether the service is currently running properly, such as with delayed parameters, or interval intervals, and make a request at the appropriate time to know whether the application is ready/healthy.
Spring Cloud traffic governance capability replacement
API Gateway
Most users will be faced with an important question: How does the flow governance work that has been done on Spring Cloud migrate to the Cloud? For example, API Gateway, The Spring Cloud Gateway provides many capabilities that allow you to do a lot of development through filter to complete your own business logic. How is Kubernetes implemented? It took a bigger scene and opened up the whole ecology. There are many tools in this open source ecosystem, such as Kong, Tyk, Gloo, and so on in API Gateway.
Take the Ambassador Edge, one of the hottest products of the moment. It natively provides authentication, distributed tracking, multi-protocol, rate limit, and more. But implementing these features in the Spring Cloud architecture requires a lot of work. Spring Cloud Gateway costs more than open source gateways such as Ambassador.
Service Mesh
Service Mesh is another, more exciting topic, and one that is currently at the forefront of research.
Communication between traditional applications has always been a complicated problem. For example, Spring Cloud Ribbon does a lot of safe and decentralized work, which has very little relevance to the business itself. So can these abilities be extracted? The community came up with a new answer: Service Mesh.
What the Service Mesh does is intercept all traffic between nodes through a Proxy layer and forward traffic between nodes through specific gateways. Since all traffic is hijacked, you can do a lot of work, including load balance, grayscale publishing based on lable, etc.
The default setting of Spring Cloud native cannot achieve full-link gray scale, so the Load Balance policy needs to be changed, which will increase the development workload of same-origin data. But in cloud native systems, Istio can be done directly with a VirtualService. While some of Istio’s features are still in development, Istio is easier to use because it strips out all of the non-business related properties and is less tied to the application.
Two-way TLS
For example, if you want to provide two-way TLS, in a traditional application you have to send an email notifying yourself that you want to upgrade to two-way TLS, and then everyone has to get their own certificate, which is a painful process. Now with the Service Mesh, a one-line declaration can force two-way authentication between different proxies to ensure network security. Security between the Service and Proxy is ensured by isolation.
fusing
The destinationRule capability provided by Istio can also be used directly with fuses featured in the Spring community by simply configuring parameters such as the maximum number of connections to access, the number of times an error is rejected, and the interval between half-open retries.
Centralized metircs
Metrics can be captured by Sidecar, rather than by applications as in traditional architectures. If the application itself still exposes some business metrics, these are available through Promerthus’s custom capture. However, if you just want to know some network throughput metrics, the application itself does not need to provide them directly, which reduces the burden on the application business.
Summary and Prospect
The emergence of Service Mesh raises a whole new question: Do we really want to put so much middleware functionality in the application itself? It just so happens that the community is thinking about it. There are some new posts in the CNCF community recently, proposing a multi-Runtime Microservice Architecture, which is a new concept that we need to provide four capabilities around the business:
- Life cycle management: Managing when applications are started, when applications are shut down, etc. Including packaging, health check, deployment, expansion, and configuration.
- Network management: including service discovery, A/B test, gray publishing, fuse, point-to-point communication, pub-sub, etc.
- Status management: Includes workflow management, caching, and application status.
- Binding: Includes data transfer and protocol conversion.
With these capabilities, developers can focus only on business logic, and r&d efficiency can be greatly improved.
These capabilities are also available in cloud native systems. For example, life cycle can be done based on Kubernetes, network can be done based on Istio, State management can be done based on Cloud State, binding can be done based on Camel. Put these things together and the business unit doesn’t have to worry about them anymore. In order to solve the complex dependency problem, Spring Cloud requires Maven dependencies, which depend on many components. Of course, these things can be removed over time, and we only need to focus on the core part of the business unit, the business logic, because this is the only part of the real dynamic logic.
The complexity of a single architecture is positively correlated with scale. The larger the scale, the more complex the middleware. FaaS, in the process of increasing complexity, has increased the scale but the complexity is the opposite. Mecha is designed to keep complexity at a low level as it scales.
Finally, looking a little further ahead, Istio is still in development. Istio V1.9 has been speeding up since then, everything is for production. The entire community is actively pushing the Sidecar model to move out a lot of things that are not business units, such as load balancing, service discovery, elastic scaling. As for whether we can go to the multi-runtime route in the future, is also our outlook, we hope that you can discuss with us the future of the whole cloud native architecture.
Q&A
Q1: Does Istio have a good controller? At present, there is no mature controller, and it is difficult to use Istio.
A: It’s true that Istio doesn’t have A good controller, because it still seems to be A cutting edge direction. Known ones like Solo. IO, they’ve made some products that are directly based on Envoy, not Istio. But I think the community is constantly evolving and the product is constantly coming out, and it’s probably going to have to be handled by someone later. There really is no good controller at the moment.
Q2: Are new cloud-native platforms and microservices capabilities language independent? What about the selection?
A: The whole CNCF community beats the Spring Cloud community because of language neutrality. If the language is bound, many platforms can be self-consistent, which is simpler. But we have to accept the reality: in the current Internet system, or the software system as a whole, heterogeneity has become an inevitable trend. It is impossible for us to require everyone to know only one language. At this time, different languages can be used to achieve different things, and different ecosystems will have different compositions. Kubernetes and the CNCF community are doing just that.
Q3: Can Kuber-Proxy completely replace Spring Cloud Zuul or Gateway?
A: That’s actually an interesting question. Kube-proxy still does forwarding work based on iptables and IPVS, although Cillium’s KUbe-proxy eBPF does a lot of work. As for whether it will be integrated into Kubu in the future, from my experience, it is mainly because Service Mesh integrates many business attributes, which may not be what Kube-proxy wants to support.
Q4: Under Kubernetes environment, how can developers easily debug local code?
A: There are standalone versions of Kubernetes, such as Minikube or some cloud vendors, which provide reasonable local direct access to cloud services. I would recommend that developers try Minikube/K3s, which runs locally and is very convenient for some debugging.