To address the usual challenges of microservices such as staggering operational overhead, repetitive effort, testability, and the benefits of microservices such as code decoupling and ease of horizontal scaling, ZStack builds an in-process microservice architecture by including all services in a single process, called a management node.
motivation
Building IaaS software is difficult, a lesson that has been learned from existing IaaS software on the market. As integration software, IaaS software typically needs to manage complex subsystems (such as virtual machine manager hypervisors, storage, networking, authentication, etc.) and needs to organize and coordinate interactions between multiple subsystems. For example, vm creation involves the cooperation between VM management modules, storage modules, and network modules. Since most IaaS software is often too quick to start solving a specific problem without thinking through the architecture, their implementations often evolve into:
As a software grew, the monolithic architecture eventually became such a monolithic mess that no one could modify the system’s code without building the entire system from scratch. This monolithic programming problem is the perfect place for microservices to step in. By dividing the functionality of the entire system into small, specialized, independent services, and defining the rules for interaction between services, microservices can help transform complex, cumbersome software from a tightly coupled, mesh topology to a loosely coupled, star-shaped topology.
Because services are compile-independent in microservices, adding or removing services will not affect the architecture of the entire system (although removing some services will result in a loss of functionality). There is much more to microservices than we have already discussed: microservices do have a number of compelling advantages, especially in a DevOps process that involves many teams in a large organization. We’re not going to discuss all the pros and cons of microservices, we’re sure you’ll find plenty of articles online, but we’ll focus on some of the features that we think will have a profound impact on IaaS software.
The problem
While microservices can decouple architectures, this comes at a cost. Read Microservices – Not A Free Lunch! “Failing at Microservices” and “Failing at Microservices” will have a deeper understanding. Here, we highlight some of the things that we think have a major impact on IaaS software.
1. Difficulty in defining service boundaries and repetitive work
One of the challenges of creating a Microservices architecture is deciding what part of the code should be defined as a service. Some are obvious, for example, that the logical code that handles the host part can be defined as a service. However, the code that manages database interactions is very difficult to determine whether or not it should be defined as a service. Database services can bring clarity to the overall architecture, but this can lead to serious performance degradation. Typically, code like this can be defined as a library, which can be invoked by individual services. Since all services are generally developed and maintained in separate directories, creating a virtual library that provides interfaces to different single pieces of software requires developers to be able to communicate well with different groups of developers. In summary, the service is easy to duplicate the wheel and leads to unnecessary duplication of work.
2. Software is difficult to deploy, upgrade, and maintain
Services, especially those spread across different processes and machines, are difficult to deploy and upgrade. Users often have to spend days or even weeks deploying a fully operational system and are afraid to upgrade a stable system that has already been built. Although some configuration management software like Puppet alleviates this problem to some extent, users still have to overcome a steep learning curve to master these configuration tools just to deploy or upgrade a piece of software. Managing a cloud is very difficult, and efforts should not be wasted on managing software that is supposed to make life easier. The number of services does matter: IaaS software typically has many, many services. Take the famous openstack for example. In order to complete a basic installation you will need: Nova, Cinder, Neutron, Horizon, Keystone, Glance. In addition to Nova, which needs to be deployed on every host, if you want 4 instances, and each service runs on a different machine, you need to operate 20 servers. Although this artificial case is unlikely to happen in real life, it still illustrates the challenges of managing isolated services.
3. Scattered configurations
Services running on different servers maintain copies of their configurations scattered around the system. System-wide configuration updates are often done by ad-hoc scripts, which can lead to inexplicable failures caused by inconsistent configurations.
4. Additional monitoring efforts
To track the health of the system, users must make extra effort to monitor each service instance. These monitoring software, either built by third-party tools or maintained by the service itself, still suffer from the same problems that microservices face because they are still software that works in a distributed fashion.
5. Plugin killers
The word plug-in is rarely heard in the microservice world because each service is a small function unit running in a different process; The traditional plug-in pattern (see The Versatile Plugin System) aims to hook up different functional units to each other, which seems impossible to microservices, and even anti-design patterns. However, for business logic that is natural to impose tight dependencies between functional units, microservices can make things very bad, because without plug-in support, modifying business logic can trigger a cascade of service modifications.
All services are in one process
Recognizing all of the above and the fact that a working IaaS software must run with all of the orchestration services, ZStack encapsulates all of the services in a single process called the management node. In addition to some of the benefits that microservices already offer, such as decoupling architectures, in-process microservices give us a number of additional benefits:
1. Concise dependencies
Because all services run in the same process, the software only needs a copy of the supporting software (e.g. Database Library, Message Library); Upgrading or changing the support library is as simple as what we would do for a single binary application.
2. High availability, load balancing and monitoring
Services can focus on their business logic without the distractions of high availability, load balancing, and monitoring that only the management nodes care about; Further, state can be separated from a service to create a Stateless service, as described in ZStack’s Scalability Secrets Part 2: Stateless Services.
3. Centralized configuration
Since all services in a process share a configuration file — zstack.properties; Users do not need to manage various configuration files scattered across different machines.
4. Easy to deploy, upgrade, maintain, and scale horizontally
Deploying, upgrading, or maintaining a single management node is just as easy as deploying and upgrading a single application. Scale-out of services is as simple as adding management nodes.
5. Allow plug-ins
Because it runs in a single process, plug-ins can be created just as easily as adding plug-ins to a traditional single-process application. In-process microservices are not a new invention: Back in the 1990s, Microsoft defined servers as remote, local, and in-process in the Component Object Model (COM). These in-process servers are DLLs that are loaded by applications in the same process space and are microservices in the process. Peter Kriens claimed four years ago to have defined a service that always communicates within the same process, OSGi µ Services. In microservices, a service is usually a logical representation of a repeatable business activity that is disconnected, loosely coupled, self-contained, and a “black box” to the consumer of the service. To put it simply, a traditional microservice usually only cares about specific business logic, has its own API and configuration methods, and can operate as a standalone application. Although ZStack’s services share the same process space, they share most of these features. ZStack is largely a project written in the strongly typed Java language, but there are no compile dependencies between the orchestration services, for example: Computing services (including VM services, host services, region services, and cluster services) are not dependent on storage services (including disk services, basic storage services, backup storage services, disk snapshot services, and so on), although these services are tightly coupled in business processes. In the source code, a ZStack service is nothing more than a Maven module built as a separate JAR file. Each service can define its own APIs, error codes, global configuration, global properties, and system tags. For example, KVM hosting services have their own APIs (shown below) and various ways to allow users to define their own configurations.
<service xmlns="http://zstack.org/schema/zstack">
<id>host</id>
<message>
<name>org.zstack.kvm.APIAddKVMHostMsg</name>
<interceptor>HostApiInterceptor</interceptor>
<interceptor>KVMApiInterceptor</interceptor>
</message>
</service>
Copy the code## Configure through global configuration
Note: this is a brief overview of the global configuration. Users can use the API to update/obtain the global configuration. Here is the global configuration view.
<globalConfig xmlns="http://zstack.org/schema/zstack">
<config>
<category>kvm</category>
<name>vm.migrationQuantity</name>
<description>A value that defines how many vm can be migrated in parallel when putting a KVM host into maintenance This value defines the number of VMS that can be concurrently migrated when a KVM host is in maintenance mode.</description>
<defaultValue>2</defaultValue>
<type>java.lang.Integer</type>
</config>
<config>
<category>kvm</category>
<name>reservedMemory</name>
<description>The memory capacity reserved on all KVM hosts. ZStack KVM agent is a python web server that needs some memory capacity to run. this value reserves a portion of memory for the agent as well as other host applications. The value can be Overridden by system tag on individual host, cluster and zone level (Reserved memory capacity of all KVM hosts) The KVM agent in ZStack runs as a Web server that requires a portion of memory to run Python. This value reserves a portion of memory for the agent and other host applications. System labels on a single host, cluster, or region can override this value.</description>
<defaultValue>512M</defaultValue>
</config>
</globalConfig>
Copy the codeConfigure by global properties
Note: The following code corresponds to the corresponding properties in the zstack.properties folder
@GlobalPropertyDefinition
public class KVMGlobalProperty {
@ GlobalProperty (name = "KvmAgent agentPackageName", defaultValue = "KvmAgent - 0.6. Tar. Gz)"
public static String AGENT_PACKAGE_NAME;
@GlobalProperty(name="KvmAgent.agentUrlRootPath", defaultValue = "")
public static String AGENT_URL_ROOT_PATH;
@GlobalProperty(name="KvmAgent.agentUrlScheme", defaultValue = "http")
public static String AGENT_URL_SCHEME;
}
Copy the code## Configure via system tag
Note: The following code corresponds to the corresponding system label in the database.
@TagDefinition
public class KVMSystemTags {
public static final String QEMU_IMG_VERSION_TOKEN = "version";
public static PatternedSystemTag QEMU_IMG_VERSION = new PatternedSystemTag(String.format("qemu-img::version::%s", QEMU_IMG_VERSION_TOKEN), HostVO.class);
public static final String LIBVIRT_VERSION_TOKEN = "version";
public static PatternedSystemTag LIBVIRT_VERSION = new PatternedSystemTag(String.format("libvirt::version::%s", LIBVIRT_VERSION_TOKEN), HostVO.class);
public static final String HVM_CPU_FLAG_TOKEN = "flag";
public static PatternedSystemTag HVM_CPU_FLAG = new PatternedSystemTag(String.format("hvm::%s", HVM_CPU_FLAG_TOKEN), HostVO.class);
}
Copy the codeThe service declares itself in the XML file of Spring’s bean. For example, a partial declaration of KVM looks like this:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:zstack="http://zstack.org/schema/zstack"
xsi:schemaLocation="Http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd http://zstack.org/schema/zstack http://zstack.org/schema/zstack/plugin.xsd"
default-init-method="init" default-destroy-method="destroy">
<bean id="KvmHostReserveExtension" class="org.zstack.kvm.KvmHostReserveExtension">
<zstack:plugin>
<zstack:extension interface="org.zstack.header.Component" />
<zstack:extension interface="org.zstack.header.allocator.HostReservedCapacityExtensionPoint" />
</zstack:plugin>
</bean>
<bean id="KVMHostFactory" class="org.zstack.kvm.KVMHostFactory">
<zstack:plugin>
<zstack:extension interface="org.zstack.header.host.HypervisorFactory" />
<zstack:extension interface="org.zstack.header.Component" />
<zstack:extension interface="org.zstack.header.managementnode.ManagementNodeChangeListener" />
<zstack:extension interface="org.zstack.header.volume.MaxDataVolumeNumberExtensionPoint" />
</zstack:plugin>
</bean>
<bean id="KVMSecurityGroupBackend" class="org.zstack.kvm.KVMSecurityGroupBackend">
<zstack:plugin>
<zstack:extension interface="org.zstack.network.securitygroup.SecurityGroupHypervisorBackend" />
<zstack:extension interface="org.zstack.kvm.KVMHostConnectExtensionPoint" />
</zstack:plugin>
</bean>
<bean id="KVMConsoleHypervisorBackend" class="org.zstack.kvm.KVMConsoleHypervisorBackend">
<zstack:plugin>
<zstack:extension interface="org.zstack.header.console.ConsoleHypervisorBackend"/>
</zstack:plugin>
</bean>
<bean id="KVMApiInterceptor" class="org.zstack.kvm.KVMApiInterceptor">
<zstack:plugin>
<zstack:extension interface="org.zstack.header.apimediator.ApiMessageInterceptor"/>
</zstack:plugin>
</bean>
</beans>
Copy the codeThe management nodes, which act as containers for all services, read their XML configuration files during startup and load each service.
conclusion
In this article, we demonstrated ZStack’s in-process microservices architecture. By using it, ZStack has a very clean, loosely coupled code structure that is the foundation for creating strong IaaS software.