6. 5. Smart Mattress

0.1 background

Ali overtime add tired of me, then choose state-owned enterprises, as the next station of life.

IT is well known that, disregarding some incongruity, there are only two most profitable industries in China today: IT and finance.

So, smart me, since has done IT, so next, of course, to do financial small…… ! — Oh no, IT in finance.

So I went back to Xi ‘an and started working as a front end team leader (please don’t get me wrong, not that one) in a trust company, a somewhat promising job.

0.2 Environment at that time

It is a trust company and a state-owned enterprise.

As we all know, the technical system of most state-owned enterprises is very general. But fortunately, from the top leadership, to the department heads, are very capable and courageous. That’s why I came to this company. After all, although coming to a state-owned enterprise is for stability, it doesn’t want to dawdle.

When I first came to this company, I was faced with the mess left by my predecessor. Specifically, there are the following:

  1. There are 3 front-end team members. The advantage is a good attitude, but the disadvantage is that the technical level is average, especially the code habit is not good.
  2. One front-end project is left behind, but the problem is very big. The specific situation will be described below.
  3. The project, which was expected to last three months, has not been completed for five months.
  4. The interface provided by the back end changes frequently and often without notice, causing the code to suddenly fail to run;
  5. Others, slightly;

In particular, this front-end engineering issue:

  1. All back-end interfaces are cross-domain. On the one hand, the back-end is open cross-domain. On the other hand, when accessing the back-end interface, the front-end uses an absolute path.
  2. There are four environments (development/test/preproduction/production), and packaging to different environments requires four different commands (remember rule 1, the absolute path of the interface URL), which means that there are four packages;
  3. There is no automated deployment (CI/CD), need to manually package, and then copy the code, (theft: author: 00 water, wechat: QQ20004604) through FTP upload to a different server. So it’s easy to misoperate;
  4. Little else.

Basically, it’s just a mess, nothing to write home about.

The only thing to be thankful for is that the leadership’s support is not bad. Although outsourcing development can only be recruited, I have full power to recruit. If I say yes, I will recruit.

0.3 last blog post

In May, 2013, I wrote an 8000-word brief Introduction to the Front-end Architecture of Large-scale Projects, which reached the third place in the history of Nuggets (nearly 12W reading and 3500+ likes). But there are some people feedback, feel that said a little shallow, the lack of specific implementation methods.

But that article, even if it is relatively shallow, has already 8000 words, this article compared with the last blog, detailed several points, I wrote this sentence when calculating, about 8000 words.

So it’s not that I don’t want to talk more deeply, but I want to write more deeply, each point may be more than 8000 words, try my best.

1. Resolve legacy issues

1.1 train of thought

Before you do something, you must have a clear idea. A few principles:

  1. Ensure existing stability;
  2. Do things with low cost and high yield first;
  3. Consider long-term costs as low as possible;

Based on the above situation, I judge that:

  1. The most important factor is the human factor;
  2. Secondly, judge the priority of the problem and solve it one by one.
  3. Finally, build and optimize the front-end technology system in daily life, and try to improve the quality of our code through tools and specifications.

Although I can solve all the above problems by myself and do it well, I can’t finish all the projects by myself. No matter how competent I am, as the team expands, the marginal benefit is bound to get lower and lower.

1.2 Get to know your team members

Therefore, I made an appointment to chat with each student individually to understand their thoughts and personal situation and prepare for the following actions.

In the process of the conversation, I first understand their ideas and personal situation, based on their personal wishes, put forward ideas for their future, and give specific feasible development direction.

The overall results of the conversation can also, at least on the surface of everyone (anti-theft: the author: 00 water, wechat: QQ20004604), we have ideas and drive (finally verified, indeed). There’s no need to worry about deliberate procrastination.

[OK, human problem solving]

1.3 Analysis of priority and urgency

Analyze and judge existing problems:

The problem severity Repair cost
Interface changes High, seriously affecting the schedule Low, as long as it is well coordinated with the back end
The lack of CI/CD In the Low, need to write a script to automatically synchronize code from Gitlab
Scaffolding of low quality In the In, it has great influence on old projects
Interface cross-domain high Existing code, new project low
Member code is not standard high High, it takes time
I forgot to list it * *

[Existing problems are clear]

1.4 Define architect tasks

The core purpose of front-end technology construction for front-end architects is to improve development efficiency, ensure development quality, and ensure high-quality and timely delivery of projects.

At the same time, considering the actual situation of medium and long term research and development, combined with the actual ability of the team, to make technology reserve for the future and provide more possibilities for business development.

So, divide your tasks into three categories:

  1. Infrastructure design: The main purpose is to avoid common problems and improve development efficiency through process design from the framework level.
  2. Engineering design: strongly related to code, the main purpose is to improve code quality, enhance the long-term maintainability of code, reduce development time and cost;
  3. Team management: through reasonable and effective team management, improve the team man-efficiency ratio, for the future project research and development, technology development, talent reserve, technology research and development;

I want to approach it from these three perspectives, both to solve existing problems and to support the long-term future.

[Clear direction]

So, began to work, with four months, to do the following work.

2. Infrastructure design

2.1 the front-end pipeline

The core design of infrastructure is process design. A good process can run efficiently and at low cost, while avoiding a lot of problems through process.

The basic process design is as follows:

In this process, the code is divided into three layers: personal code — main branch code — online code, which is divided into dev, QA and Pro versions.

Developers only have control over their personal code and cannot directly influence changes to the main branch code. When submitting to the main branch, they need to initiate a merge request and merge the code into the main branch after the administrator reviews the code.

And the main branch code and online code isolation, (theft: author: 00 water, wechat: QQ20004604) by the administrator to the specified version of the code released to the specified environment. Whichever dev, QA, or Pro it is, will pull the specified commit directly from GitLab and then package it for publication.

This secondary isolation results in a controlled, clean, and stable code environment. Therefore, regardless of the environment, the release of code is idempotent, in other words, stable, controllable and predictable.

Through the above process, the front-end code can be guaranteed to run in the server side with high quality and high stability.

2.2 Code version control

In traditional front-end separation projects, front-end code is usually deployed independently by the front-end itself. After packaging, front-end code is directly deployed to a specified directory. But there are several serious problems with this process:

  1. Cache problem: In order to improve user access speed and reduce traffic pressure caused by loading resources, we usually cache front-end static resources. The downside, however, is that when we need to publish or roll back versions, users will inevitably access cached static files.
  2. Slow release and rollback: In traditional scenarios, release code needs to be repackaged every time release and rollback is required. (the author: zero zero | water, wechat: qq20004604) this process on the one hand, slowly, on the other hand, not necessarily accurate packaging to a specific one (not accurate).

Therefore, according to the actual situation, use our own front-end Pipline management. The core has the following points:

  1. Code versioning: For code that needs to be submitted to the online server, it is tagged in GitLab, and only the code with the correct tag is packaged and published online. The key to this operation is that static resource files, such as CSS and JS, are stored in folders with their version numbers and multiple versions exist simultaneously. HTML does not cache, but serves as an entry point for importing different versions. Example directories are as follows:

In the Nginx layer, HTML is cache-free and loaded with the latest one each time it is loaded. Other static resources serve as the cache and expire in 15 days. See the following figure for the release process. When switching the version, the details are as follows:

2. Version coexistence, owing to the relatively small size of the static file itself resources (due to do out of the public static files, so the smaller), (the author: zero | zero water) currently wealth phase ii after packaging only 16 MB, the future, after the second optimization will be reduced to about 10 MB. So even if multiple versions are on the server at the same time, it doesn’t take much.

The control method is that the entry HTML file is unique, by modifying the version number of the static resource file pointed to in the HTML, to specify the version number that the user is currently accessing. The user access flow chart is as follows:

As you can see from the figure above, static resources are only loaded on the user’s first access. The local cache is accessed in other times, which greatly improves the access speed and user experience and reduces the I/O pressure on the server network.

HTML does not cache and prevents users from accessing the pages of the old version after the new version is released, which leads to service processing exceptions.

At the same time, because each version number is strongly coupled to the COMMIT, code with the same version number in any environment must be idempotent, so you don’t need to worry about code behaving differently in different environments.

2.3 Private NPM sources

Since we usually access the Intranet, we need to consider the issue of Intranet isolation in the future. Therefore, build a private NPM source to solve the problem from the basic design level ahead of time. The private source uses the open source technical solution Verdaccio. The advantages are as follows:

  1. Lightweight: Verdaccio is developed based on Node.js, with less occupation of resources, fast running speed and simple configuration, which is suitable for the actual situation of our company.
  2. Low configuration difficulty: You can quickly configure user groups, permissions, running ports, and storage addresses, which is sufficient for common scenarios.
  3. Ensure dependency version stability: Placing dependencies locally is more stable than placing dependencies in third-party remote sources. On the one hand, it can improve download speed (because some third-party dependencies are unstable to connect to the Internet). On the other hand, it can ensure that dependencies always exist (cached in local private NPM sources).

Through the combination of the above three points, the uncertainty of published online code is solved from the architecture level, and the reliability of front-end code is improved.

In addition, our front-end group will release some self-encapsulated NPM packages to its OWN NPM source and provide them to different projects for use, so as to avoid the r&d waste caused by repeated development and the unreliable public dependence caused by the unstable capabilities of developers.

2.4 Public Static Resource Cache

In addition to the business code, the front end also has some common static resources, such as Vue JS files, ElementUI resource files, Echarts resource files, and some image files.

These files are shared by all projects. If these files are scattered in each project, it is unnecessary and easy to lead to the inconsistency of files depending on different projects. In addition, these files will be loaded repeatedly, wasting network bandwidth and static server storage space, and making no sense.

Therefore, setting a public static path can improve access efficiency and performance by configuring it in scaffolding and directly loading these resources to the specified nginx static server and doing long caching.

In the future, a special static resource management service will be made for transferring static resources (mainly images) to the static resource service and CDN acceleration, and then the front-end engineering will use these resources by introducing image urls directly. This can reduce the network bandwidth consumption of your own server.

2.5 other

All of the above is realized, and there are a few others, but too trivial – in fact, mainly BECAUSE I can not write, I will not write.

3. Engineering design

3.1 Custom Webpack scaffolding

In order to improve the code quality with low cost and high efficiency, based on the actual situation of the company, a customized Webpack scaffolding was developed to solve the following problems:

  1. Avoid unlimited third party libraries: Unlimited third party libraries (vue+jquery, etc.) is an often overlooked but problematic behavior. Because developers often don’t read the source code of third-party libraries, they can’t be guaranteed to be secure. Custom scaffolding, which provides all the dependencies needed for development and strictly controls the introduction of new dependencies, ensures that the entire project is safe. (Anti-theft: Wechat: QQ20004604)
  2. Constraining developer code specifications: The most common problem with development is that code specifications are disorganized and written in isolation, resulting in high maintenance costs. Eslint solved these problems by introducing a mandatory code style and automatically reporting errors for code that did not conform to the specification. And since the IDE supports automatic formatting of code by applying esLint rules. It also reduces the constraint cost to developers and reduces the difficulty of implementing code specifications.
  3. It is easy to provide standard scaffolding to other developers and provide technical support (such as a technology stack that can unify the large front end).

The scaffolding I use is my own vue scaffold based on webpack5 configuration, including a variety of configurations.

3.2 Integrate the debugging console

Consider the large amount of form content in the project’s pages. To facilitate the development of test syndication and avoid having to manually enter many forms each time, a dedicated debugging console is integrated. When you open the debug console with a special operation, a small floating window appears on the page with preset buttons that you can click to perform predefined actions (such as filling out a form).

Click the Test button as shown below and the form will automatically fill up:

With the integrated debugging console, there are the following benefits:

  1. Reduce the inefficient behavior of repetitive operations: auto-filling forms is the most common scenario. When infinite pull-down loading is required, the specified page number can be directly loaded or the number of pages loaded can be increased, so as to quickly obtain the data of a page without modifying the business code;
  2. More debugging information: Sometimes it is necessary to display more information on the page than the user can use, but it is not elegant to modify the original business code. So turn this extra information on or off through the debug console. And since it is off by default and local, it can also be used for online debugging;
  3. Online debugging: after the embedded point system is added in the future, when the user encounters an abnormal situation and cannot reappear, the user can be taught to open the debugging console, and then click the button to push the user history operation. You can then analyze the user behavior on the server side to find out the cause of the problem.

3.3 Multi-page master-slave design

According to the actual business needs of the company — workbench as the core platform, other pages access, and the backend is micro service design. Therefore, the design idea is multi-project multi-page master-slave management. Specifically, there are the following points:

  1. Multi-engineering: Considering that we will deploy in the form of multi-system access with the workbench as the core in the future. So break up the front end, with the workbench as the core entry point, and then the story, the developers, the system, and so on, into small and medium-sized front end projects. This split reduces the difficulty of code development and maintenance, and reduces the cost of collaboration between different story development.
  2. Multiple pages: For a long-term maintenance project, long-term maintenance difficulty must be considered. Therefore, based on my previous experience of stomping holes and maintaining historical projects, I designed the front page to be multi-page mode. Specifically, each story is divided into different pages according to function points, and the data exchange between pages is carried out by local cache, browser parameters, etc. With this isolation, the cost of adding new features to a new page, refactoring a single page, or experimenting with new technologies on a new page is very low;
  3. Master-slave management: interventional engineering. Other pages that need to be opened in the workbench need to be pre-registered after the page ID to call open. In this way, the system security can be improved through centralized control to prohibit third-party web pages from opening illegal pages on the workbench.

Through such a design, the traditional large-scale front-end engineering can be divided into micro-front-end engineering similar to back-end micro-service to improve the development efficiency, especially the parallel development efficiency. Also enhanced permission management to ensure code security.

The specific design structure is as follows:

The interaction flow between them is shown below

No matter how it jumps behind, because it is forwarded by Nginx, users are apparently accessing the same URL without affecting user cognition.

4. Team management

4.1 Permission Control

The permission control here mainly refers to the code permission control, the purpose is to ensure the code security, problem control can avoid traceability.

Specific management measures are as follows:

  1. Ensure the stability of dependent versions: the code belongs to the property of the company, so based on GitLab, the code is isolated by permission, and all access permissions are closed by default. For each project, specific permissions are assigned to the development according to actual needs. Can avoid the company property loss due to code leakage;
  2. Commit permission: Allows development to be committed on its own branch, but when it comes to merging the main branch, the merge must be submitted, and then committed to the main branch after code review by the team leader. In this form, the main branch’s code must be controlled and not introduce serious problems;
  3. Publish permission: For the code to be published to dev, QA, and Pro, the permissions are converged to the group leader and only the group leader is allowed to publish the code. The benefits are that, on the one hand, you can provide a reliable and controllable version of the online code, and on the other hand, you reduce the skill requirements of the developer.

4.2 Interconnection control of Front and Rear End Interfaces

Back in October and November of last year, there was a serious problem with the front and back end development team, that is, when the back end interface changes, the corresponding front end developers and testers would not be notified beforehand and afterwards. Development is very inefficient, and there are all kinds of exceptions.

Therefore, by sorting out the development process, it is mandatory for the backend to provide interface documents in front of it and to have a controllable and stable interface document as the docking standard.

Since December, there has been a rough 80% reduction due to bugs and development and test blocking caused by temporary interface changes.

However, in practice, there are still some cases where the interface documentation does not match the actual interface, causing interface tests to be redone, and we need to try to reduce these problems.

4.3 Personnel Management

Work is done by people, so relatively speaking, good people, sometimes more important than doing things. Therefore, in order to ensure the reliability and stability of personnel, proceed from the following perspectives:

  1. Practical entry interview questions: the combination of theoretical questions and practical business questions is adopted to ensure that the onboarding personnel really understand business development and avoid personnel capability risks. After passing the preliminary interview, I will arrange to do a pen test with a small amount of code but certain difficulty, which tests my problem-solving ability and attitude. If I am willing to do it and can do it, it is basically reliable according to my current experience.
  2. Mentoring: After the new person comes in, arrange for someone to lead him or her, answer frequently asked questions, and make a story with him or her. Compared with the direct management of training by the group leader, it is more conducive to the integration of new members into the team; (Anti-theft: author: zero-zero water, wechat: QQ20004604)
  3. New person adaptation: the common problem of new people is not clear their own positioning. Therefore, the team leader is responsible for arranging the development direction of new members. In the first week of their entry, they are first arranged to develop the daily system and understand the scaffolding, and then they are arranged to do the optimization based on the existing page to help them understand the story that different people are responsible for.
  4. Clear responsibilities: Identify and inform the position of everyone on the team. Assign tasks suitable for team members according to their different abilities and attitudes. For example, Cheng Yaxu is responsible for simple pages and self-test string stories, Ma Qian is responsible for the core story of the business, And Wang Chao is responsible for the content that requires certain technical innovation;

5. Future plans

I’ve actually done a few others, but I’m not going to write about them because I don’t have enough space. (Mainly because I can’t write. I’ve been writing for a week.)

1. Static image resource server:

At present, the image is packaged into the project, but this disadvantage is too heavy engineering, the second is the nGINx server pressure. Therefore, we need to consider building a special static resource server in the future. After uploading, it will be automatically transmitted to CDN and return image links on CDN. Then introduce image links directly into the page;

2. Backend renders HTML:

This is the core focus of my future work. A better alternative to nginx rendering HTML is Backend rendering HTML. The benefits are as follows:

  1. Reduce nginx server stress;
  2. Version release, from the need to manually execute scripts to replace HTML, to only need to change configuration can change the front-end code version;
  3. Can be in the form of middleware, embedded some public JS, CSS, such as buried JS;
  4. Can solve the CSRF problem;
  5. Strongly restrict web page entry;

The disadvantage is that the Backend layer needs to be developed and may require modification of the code.

  1. Gitlab integration with ESLint:

Even inheriting ESLint doesn’t work best, as opposed to requiring developers to check their own code quality. Therefore, it is best to integrate ESLint into CI/CD. – EsLint reviews code automatically when developing and submitting code to GitLab, and if the review fails, it will be called back;

  1. Buried point system:

Our common problems are the inability to know user habits and the inability to catch errors. Therefore, buried point system is very necessary. After embedding the embedded point system, it can count all aspects of the system information from the data level, including access speed, access time, the number of visitors, the distribution of the webpage visited by the number of people from different positions,

  1. CSRF:

Financial system security is very important, in addition to the general protection against XSS injection, also need to protect against CSRF (Cross-site request forgery) attacks. The solution is to avoid using cSRF-token.

The end:

In general, some front-end related technology construction has been done, but more needs to be built in the future, especially to make it automated, GUi-oriented, convenient and centralized.

For example, the self-built OA system can access Dingding attendance and GITLab to automate the whole process (one-click release, push to the server, check reliability, and notify the relevant person in charge), and automatically alarm and notify the relevant person in charge if there is an online problem.

And we’ve been on K8S for the last 2 months (this was written before we started, so it’s only up to the end of k8S), more on that in the blog below (at least six months).

There’s a lot of work to be done in the future, and there’s a lot to be done for front-end architects.