Why develop such a platform
When we want to write a Node.js application, we have to deal with a lot of things (machines, environment, deployment, etc.) other than writing code, which greatly increases the cost of writing the application. The Node Labs platform will provide application creation, running environment, deployment, monitoring and other functions, making it easier to create Node.js applications.
What projects are appropriate for deployment and management on this platform
Our early goal is for internal, simpler node.js applications with low load requirements. We develop a set of specifications for deployment management applications on this platform, and all Node.js applications that meet the specifications can be deployed on this platform.
How to build a NodeJS application deployment platform
We consider using a Docker-based containerization scheme to achieve the goal of rapidly deploying Node.js applications. However, the management, communication and scheduling of containers also become issues to be considered, such as how application containers access database containers, how to define port mapping rules, how to dynamically expand capacity, and so on. Docker technology alone obviously cannot meet our needs.
Faced with these difficult problems, we chose the direction of Kubernetes.
Kubernetes can be understood as a container cluster, its purpose is to allow users to manage cloud container cluster through Kubernetes cluster, without users to carry out complex Settings. The system will automatically select appropriate work nodes to perform specific container cluster scheduling processing work.
We consider using four resources, Deployment, Ingress, Service, and Pod for our purposes. A Pod is the basic unit that Kubernetes creates or deploys. It encapsulates an application container and is an instance of a single application. Service is responsible for abstracting the functionality of a set of pods and mapping them to the Pod’s ports. Ingress is responsible for mapping domain names to services so that they can be accessed by domain names. Deployment is an abstraction of Deployment to avoid working directly with pods.
It can be said that Kubernetes solves many of the pain points of using Docker technology in production.
The overall architecture
The overall structure is as follows (to be extended)
Users submit application information (such as gitlab address, Branch, Node version, and domain name prefix) in the foreground to one-click deployment and view the deployment progress (real-time logs) and application status. The core service of Node Labs completes tasks such as application deployment and viewing through interaction with its surroundings.
We provide two types of “application” creation. In addition to Node.js applications, we also provide the creation of database containers. If database-related services are required, you can create the required database before creating the application and then deploy the application.
Core services
As shown in the figure below
UserService is responsible for user identity and permission control and is the basis for calling other services. In Node Labs platform, UserService interconnects with the login service of the internal system for identity authentication, and then some permissions will be extended.
DeploymentService is responsible for application deployment. It mainly includes image generation and deployment resource generation. Image generation includes code pull, image packaging, image push to the cloud and other parts. Generate Deployment configuration based on user input, call kubeService to generate Deployment, get Deployment information, and so on. Each deployment contains information about applications and users, facilitating subsequent query.
AppService is responsible for application status and information related, mainly through obtaining the information of Pod in Kubernetes cluster, to analyze its status.
DbService is responsible for database deployment. Currently, it provides mongo and mysql, and will provide more diversified configurations in the future. After a database is created, the system returns the connection information of the database. You can modify the configuration.
KubeService is responsible for the invocation of core services and the Kubernetes cluster. DeploymentService, AppService, and DbService all require calls to KubeService to obtain container-specific information, which is a relatively separate part.
Kubernetes resources
We choose to use RESTful API to communicate with Kubernetes cluster, which will have stronger scalability.
After the core Service generates the application image, KubeService will call the interface to generate four resources including Deployment, Ingress, Service and Pod. The parameters for generating resources are derived from the application’s configurations, which are formulated in a canonical form to prevent unexpected situations.
In the development of Node Labs platform, the above four resources can meet the basic requirements of Node.js application deployment. Of course, based on the power provided by the Kubernetes API, we can extend the core functions of the Node Labs platform through simple interface calls.
Message service
Responsible for core services and front end real-time notification push, such as deployment status, rolling logs, etc. Message queues are based on Kafka clusters. Node Labs core services will be the main content producer, but can also provide some full-duplex communication capabilities based on this architecture.
other
In addition to deployment and application status viewing, the platform also provides monitoring, log analysis, and dynamic capacity expansion functions. These perimeters are still under planning and are an integral part of the platform.
The detailed implementation of each module will be described in the next blog post, and the next one will go through the entire process of deploying an application in detail.
Creative Commons Attribution – Non-commercial Use – Same way Share 4.0 International License
