preface
The growth of Bump Lab from the original front-end team to the current full-end team means that we not only pay attention to the technical capabilities of the front-end, but also pay attention to the full-end and full-stack capabilities. In this article, from the perspective of the front end team, we will explain some of the difficulties we encountered in building the server architecture initially, the service architecture we have built, and how we can better support business growth.
A bit of deja vu
First, let’s take a look at some of the scenarios that exist in everyday work.
- Scenario A: In some services, the data interface provided by the underlying data team does not provide HTTP calls, and other server teams need to be found to encapsulate the data interface. In this case, other teams need to wait for the schedule, which may cause service failure.
- Scenario B: The front-end page performance is sluggish. Because there are too many call interfaces, the data needs to be aggregated by other server teams.
- Scenario C: We need SSR in some projects, and we need to reuse a unified set of templates at the front and back ends.
- Scenario D: We incubate some projects internally that require interface services and need to wait for support from other server teams.
Behind these scenarios, we urgently need to set up a server r&d team to undertake part of the business service development and better help the team’s future development.
The molding
In terms of team building, there are mainly two ways of “internal selection” and “external recruitment”. In terms of team development, we went through three main stages.
A prototype
In the initial phase, NodeJS was chosen as the server-side programming language for two main considerations:
- Most of the students in the team are familiar with Javascript and can learn NodeJS quickly with low cost.
- SSR has natural advantages, the front and back end can share part of the code.
At this stage, we quickly incubated a series of systems and platforms, such as Mock platform, front-end monitoring platform, bottom-of-the – world platform, etc. The main goal was to enrich the front-end r&d system, improve the development and collaboration efficiency of the front-end and back-end, and also deposit some NodeJS middleware.
growth
After a certain amount of precipitation in service development efficiency, performance, stability, security and other aspects, we began to think about how to more standardized service development, more efficient support business growth.
At this stage, we not only exported a series of knowledge systems such as “R&D specification”, “R&D process” and “development framework”, but also built relevant R&D platforms such as “deployment platform” and “general management platform”. In terms of business development, we have implemented “Tengo” game service with NodeJS, “data aggregation service” with OpenResty, and SSR in some channels, etc.
Fu can
In the middle stage of design, we have accumulated a large number of general services, such as “page”, “picture”, “compilation” related services, part of the services to other teams, for example, we have enabled the page intelligent design related services to “jianghu platform”, “smart shop” and other products.
Internally, most of the server-side team’s technology stack is predominantly Java, with inter-service calls using the group’s in-house JSF protocol. However, NodeJS is still the main technology stack in our team, providing HTTP calls to other teams, which is quite different from Java in terms of access mode, stream limiting, code reminding, etc., and can not make good use of a large number of middleware in the group.
At this stage, we introduced the Java technology stack and formed a technical system with “NodeJS + Java” as the main server-side language. For some domain services, we provide a Java version of JSF service implementation to facilitate communication and cooperation between third-party teams.
architecture
After a few years of precipitation, our team built a rudimentary architecture on the server side.
The main goal of the construction of the server r&d system is to improve the lower limit of the team code, improve the development efficiency, improve the quality of service delivery, and promote the team to grow together.
Build a set of server r&d system, mainly around 8 aspects, including RESEARCH and development specifications, research and development platform, research and development process, document construction, team management, alarm monitoring, middleware management and infrastructure.
In terms of the overall service architecture, our daily development adopts the mode of hierarchical structure, to extract the general service logic as far as possible, output more building blocks, and reduce our development and maintenance costs.
The following is a brief description of our explorations and practices in the field of server in recent years from the aspects of “business support” and “technology construction”.
Business support
Business is the foundation of the team, only in the rapid growth of business can we continue to verify and optimize our entire service system, to ensure the reliability of the overall service.
Current which service
Linglong full name linglong intelligent design platform, provide one-stop online design services: A key cutout, free cutout, commodities play belt, the size of the change, the main figure design, online advertising banner design, shop front page design, the active page design, page design, design of interactive marketing, small figure video programming, dynamic design, video advertising design, product main photo video design, poster design, public image design, DM handbill, qr code name card design Plan, logistics surface paste design, yirabao design, poster design and so on. Provide a large number of exquisite templates and free materials, free design, another enterprise area, is committed to become a business design partner platform.
Linglong intelligent page design is the assembly of all kinds of senior e-commerce industry designers, to provide one-stop professional intelligent page and small program design services platform. The whole architecture service is lightweight and modular, which makes it more convenient to expand business scenarios in the zone. The service structure is as follows:
The overall architecture is divided into four parts: Web application layer, interface layer, service layer and data layer. In this way, entrance can be unified, single point deployment makes release more convenient, and independent deployment reduces the impact of module update on the overall service:
- Web application layer: including software and other platform applications;
- Interface layer: provides gateway service, and the request of application layer is forwarded to each module through gateway and permission verification.
- Service layer: mainly divided into the following four parts:
- Service communication: asynchronous communication using MQ, RPC communication using HTTP call;
- Business modules: namely, the core logic of services. They are divided into different modules according to the functional logic. Most functions are handled independently within the modules to achieve high cohesion and low coupling.
- Basic services: support the basic functions of business modules and uniformly control users and permissions;
- Service management: Used for service assistance to improve the stability, robustness, and flexibility of services.
- Data layer: The service mainly uses MongoDB for primary storage and Redis for data caching.
The project is developed using Typescript and follows the unified interface specification to ensure the uniform style of input and output parameters. The modular design makes service operation and iteration easy. In terms of functions, it supports the plug and plug expansion of zones and scenes, making business infinite.
Data aggregation service
In e-commerce businesses, such as channels and promotion activities, the data of commodity group and advertising group are often used. There are many redundant fields in the common interface, especially in the batch query service, the whole response package will be relatively large.
We use OpenResty to implement GraphQL service. Data is loaded on demand, which can effectively reduce packet size. Automatic data backstop ensures service availability, especially in the event of slow response of underlying services during the rush period.
Construction technology
The necessary infrastructure and technology exploration is needed to better prepare the business and the team for the future.
The following illustrates some of our infrastructure on the server side around the Talos deployment Platform and the Common Management Platform.
Talos deployment platform
The Talos deployment platform is developed based on the internal JDOS platform. It mainly provides application resource management and deployment functions, solves the problems of difficult deployment, low development efficiency and high service operation and maintenance costs, and enables the r&d students to focus more on development.
The main architecture diagram is as follows:
Let’s briefly introduce the platform in terms of resource management and application deployment.
Resource management
The project grouping function facilitates the developer to manage and view the application, traffic and resource occupation under the group.
Static website deployment Static website applications can be deployed in the same back-end application. When accessing a web browser, the browser matches the corresponding file based on the domain name or the pre-route, saving resources and improving resource utilization.
In addition, there are some other features:
- Talos gateway is provided to facilitate service forwarding and mounting;
- MongoDB visual panel function is provided to facilitate the development of students to view online database, providing read-only, read and write permissions;
- Monitors the entire process, including application creation, deployment, and container adjustment. When the CPU, disk, and load exceeds the threshold, an alarm is generated.
- There are other functions such as container quantity adjustment, on-line limit when promoting, and notification.
Application deployment
Supports multi-environment deployment, such as test, pre-delivery, and production environments. Each environment has its own configuration file and configuration property fields, and supports one-click deployment, rollback, and offline operations. The deployment interface is as follows:
Supports deployment of different project types, such as NodeJS, static site, and custom deployment.
The deployment is not interrupted. The rolling update interface of JDOS is used to control flow switching, and the application container is divided into two rolling updates to ensure that the application is not interrupted during the update.
Universal management platform
In the development process, it is often necessary to hardcode some data, and most of the data will need to be updated from time to time in the future maintenance and operation. In the past, we were often caught up in these trivial changes to the data, reducing coding efficiency. There are two ways to solve this problem. First, store the change data in the database and develop the corresponding server interface for CRUD. This way we need the resources of the database, server development, gateway domain name, so it seems that the gain is not worth the loss. The second is to dynamically configure forms in the platform, define data structures, and enter data, while the platform provides a unified CRUD gateway API. Under the above background, general content management platform emerges as The Times require. Let’s take a look at some useful features!
The general management platform provides forms and data creation, which satisfies most configuration management functions. At the same time, it provides the function of authority management, which can update data for product/operation colleagues and get rid of the trouble of letting development students modify data/version release.
Data Form – Data structure definition
Why have a form instead of just storing JSON data? Imagine that in MongoDB database, if you cannot use ORM framework such as Mongoose to define data, you can only understand the meaning of table structure design through reading data or separate documents, it must be very painful.
The functions of forms are mainly divided into form field design, user rights management and form identification editing. Field design provides Schema design similar to that of relational databases. Users can create the types and default values of fields corresponding to the table structure, and even verify data through regular expressions.
Data – Data storage
The data module provides functions including data entry, data synchronization (in different environments), version management, and access to API links.
- Data entry: verify data entry according to the field rules of the form to prevent data inconsistency in the same form;
- Data synchronization: The platform provides two environments, pre-release and formal. Users can perform full or partial data synchronization in the two environments.
- Version management: Like most content management platforms, this feature is provided to prevent user error or restore old version data;
- API link: After data is entered, this link is used to access the entered data.
The function points of the general management platform are introduced above. In practice, a large number of applications have been accessed, including Linglong, Jelly, Taro, Quark and other excellent projects.
conclusion
So far, our accumulation and precipitation in the field of service side is only the tip of the iceberg, and we need to further explore and precipitation. In the future, we will focus more on service building blocks, output more reusable and enabling building blocks, and escort the business growth.
other
Thank you for your attention to concave-convex laboratory readers. In order to provide better content, I hope you can spare a few minutes to complete a small survey. Next year, the content of concave-convex public account is up to you. Click on the direct
Join the concave-Convex Laboratory open, open source, professional, loving and dream family? Click on the direct
For those readers who haven’t followed bump Lab, please follow us. We only push 4 times a month. We cherish every push and won’t let you down. Wechat search “bump lab” can be followed.
Concave-convex reveal series article address
1. Past and Future of Bump Laboratory
2. “Concave and convex Technology Revealed · Linglong Intelligent Design Platform · Dream Oriented Design”
3. “Concave and convex Technology Revealed · Deco Intelligent Code · Start the Revolution of Production and Research Efficiency”
4. “The Secrets of Concave-convex Technology · Visualization of Linglong Page · The Road to Growth and Transformation”
5. “Concave and convex Technology Revealed · Quark Design Assets · Creating high-quality Materials of the whole Matrix”
6. “Concave and convex Technology Revealed · Tide R&D Platform · Layout of NEW R&D infrastructure”
7. Revelation of Concave and Convex Technology · Taro · From Span end to Open Span Frame
8. Revelation of Concave-convex Technology · Basic Service System · Building Technology Center of service End