The Web server provided by Ada, the front-end of Zhaopin.com, can run on the server side and the local development environment at the same time, and its kernel is the Web framework Koa. Koa is known for its excellent support for asynchronous programming, but also for its middleware mechanisms. At its core, Koa is a middleware runtime in which almost all of the actual functionality is registered and implemented.

The status quo

Ada introduced a separate @Zpfe/KOa-Middleware module starting with version 1.0.0 to maintain middleware needed in Web services. This module exports all middleware separately and Web services can be registered as needed. With the continuous improvement of functions, the module gradually accumulated more than ten middleware. The @zpfe/ KOA-Middleware module can be used as follows:

const app = new Koa()
app.use(middleware1)
app.use(middleware2)
// ...
app.use(middlewareN)
Copy the code

One of the “bad smells” is that the order of execution is implicitly agreed between middleware, but the control of the order of execution is handed over to two users (the rendering service and the API service), which means that the user must know the technical details of each middleware.

The following diagram shows the coupling between the user and the middleware:

Koa middleware architecture is an onion structure, and each middleware can be regarded as a skin of the onion. The first to register is in the outermost layer, and the last to register is in the innermost layer. When executed, it is executed from the outermost layer to the innermost layer and back to the outermost layer in reverse order. The following diagram shows how the Koa middleware executes:

Every middleware has two chances to be executed, and in our scenario, most middleware really only has one piece of logic. As the number of middleware bulges, the full execution trajectory becomes too complex, increasing the cost of debugging and understanding, which is the second “bad taste”.

For this reason, we decided to refactor the @Zpfe/KOA-Middleware module to further improve its ease of use, cohesion, and maintainability.

Analysis of the

First, take a look at the exported functions and usage of @Zpfe/KOA-Middleware, and you’ll find a pattern like this:

  • The order of middleware registration is consistent between the two users;
  • Some middleware is only registered for use in API services (such as CORS and CSRF);
  • Some middleware uses different parameters or implementations between the two users (such as parsers and entry handlers);
  • There are some features that are not really middleware (such as request contexts and fuses).

This means that we can take back middleware registration rights and allow users to control the on/off status, parameters, and even implementation of individual middleware through parameters. It is also possible to extract non-middleware functionality directly into new modules.

If we look at the order of execution of these middleware, we will see that they can be classified into several different types:

  • Initializer: Responsible for initializing data or functionality (such as initialization)x-zp-request-idAnd logging);
  • Interrupters: responsible for interrupting execution (such as CORS and CSRF);
  • Preprocessor: Responsible for preparing the environment needed to process the request (such as a parser);
  • Handlers: responsible for processing requests (such as diagnosers and entry handlers);
  • Post-processor: Responsible for the cleanup (such as cleaning up temporary files and data) after the request processing is complete.

Further analysis of the middleware contained in each category reveals that their execution is also highly consistent within the category. With the exception of preprocessors and processors that need to be executed asynchronously, all of the other types of middleware can execute synchronously.

As mentioned above, Koa middleware can be executed twice, and @zpfe/ KOA-Middleware does include some such middleware (such as logging). When classifying middleware just now, such middleware was split into two parts and assigned to different categories. For example, the logging function is split between the initializer (initializing the logging function) and the post-handler (logging the end of the request). Another way to think about such a feature is to think of it as a complete set of features, but with two different types of specific features being exported. This way, we can write all the code for the logging function in the same file and export it by defining its initialization and post-processing functions as separate functions.

The principle of

After analysis, we have a clear picture of the current state of the @zpfe/ KOA-Middleware module. Now to summarize and form some useful guidelines:

  • Single responsibility principle (SRP) : Decouple non-middleware functionality;
  • Dependency inversion principle (DIP) : do not expose functional details of the middleware to the user;
  • Self-cleaning: After the request is processed, the middleware must clean up the data generated by itself.
  • Easy to test: each component can be individually tested;
  • Progressive refactoring: Refactoring in stages, each without breaking existing functionality, with the ability to release separately.

phase

Step 1: Remove non-middleware functionality

This step is relatively simple, just need to extract these non-middleware functionality files into a separate module. Note that:

  • Independent modules should meet the standard of high cohesion and low coupling;
  • Unit tests should also be extracted into separate modules and modified to meet testing standards;
  • All users switch to separate modules one by one and modify their unit tests as appropriate;
  • Control the scope of refactoring to limit changes to non-middleware and its users.

With the non-middleware functionality removed, the @zpfe/ KOa-Middleware module is now a real middleware module.

The following diagram shows the code structure without the non-middleware functionality:

Step 2: Encapsulate the registration function

Next, encapsulate a registration function as the only export to simplify the user’s code and hide the middleware details from it.

According to the previous analysis, this registration function requires parameters that allow users to configure parts of the middleware. The main logic of the new registration function is shown below:

function registerTo(koaApp, options) {
  koaApp.use(middleware1)
  koaApp.use(middleware2)
 
  if (options.config3) koaApp.use(middleware3)
  if (options.config4) koaApp.use(middleware4(options.config4))
  // ...
  koaApp.use(middlewareN)
}
module.exports = {
  registerTo
}
Copy the code

The options parameter can be used not only to control the enablement status of a particular middleware, but also to provide configuration to the middleware. Users can use the new registration function as follows:

const middleware = require('@zpfe/koa-middleware')
 
const app = new Koa()
middleware.registerTo(app, {
  config3: true.config4: function () { / *... * /}})Copy the code

Now that the middleware registration order is wrapped inside the @zpfe/ KOA-Middleware module, users just need to know how to use the registration function. If they want to add middleware in the future, it won’t affect them too much.

The following figure shows the code structure after wrapping the registration function:

It is important to note that this step only involves the main file and user of the @Zpfe/KOa-Middleware module, not any middleware changes, following the principle of progressive refactoring. After adding and updating the unit tests, you can move on to the next step.

Step 3: Refactor the initializer

According to the previous analysis, there are several types of middleware, of which initializers are the first. The middleware contained in the initializer should be registered and managed by itself. The main logic of the initializer is shown below:

function register(koaApp, options) {
  koaApp.use(middleware1)
  // ...
  koaApp.use(middlewareN)
}
 
module.exports = register
Copy the code

This looks like a copy of the @zpfe/ KOA-Middleware module’s main file. Next, modify the @Zpfe/KOa-Middleware module’s main file to register initializers one by one instead of using initializers for unified registration:

const initiators = require('./initiators')
 
function registerTo(koaApp, options) {
  initiators(koaApp, { configN: options.configN })
 
  if (options.config3) koaApp.use(middleware3)
  if (options.config4) koaApp.use(middleware4(options.config4))
  // ...
  koaApp.use(middlewareN)
}
Copy the code

From now on, the main file of the @Zpfe/KOa-Middleware module only interacts with the initializer, not with the multiple middleware that the latter contains. That is, we have hidden the logical details of the initializer middleware from the outside. When the logic is refactored further, it is not beyond the scope of the initializer.

The middleware contained in the initializer executes synchronously and can be reduced to functions, organized into a queue of functions, and executed sequentially. The modified initializer is shown below:

const task1 = require('./tasks1')
const taskN = require('./tasksn')
 
function register(koaApp, options) {
  const tasks = []
  if (options.config1) tasks.push(task1)
  // ...
  if (options.configN) tasks.push(taskN)
 
  async function initiate (ctx, next) {
    tasks.forEach(task= > task(ctx))
    return next()
  }
 
  koaApp.use(initiate)
}
Copy the code

All of the initializer type middleware is simplified to a synchronization function that creates a task list based on the parameters passed in when registering, and then registers itself as a middleware that executes the task list sequentially.

After the unit tests are supplemented and updated, the refactoring of the initializer is complete. In this step, we merge multiple middleware pieces into one piece and encapsulate their logic within it, which makes the code for the @Zpfe/KOA-Middleware module more structured and easier to maintain.

The following diagram shows the structure of the code after refactoring the initializer:

If we review all the refactoring in this step, we will see that the user is not involved, which is the benefit of hiding the internal logic from the outside during the second step of the refactoring. Similarly, we did not make any changes to the non-initializer middleware, which is outside the scope of the refactoring in this step and will be refactored in subsequent steps.

Step 4: Refactor the remaining middleware types sequentially

Once the reconfiguration of the initializer is complete, you can follow the same logic to refactor the remaining middleware types in turn: interrupters, preprocessors, processors, and post-processors.

The code structure after these refactorings is shown below:

Again, it is important to control the scope of the refactoring, and complete one type of refactoring (including unit tests) before moving on to the next type.

Step 5: Take a holistic look

Now the refactoring is nearing its end. For users, the @Zpfe/KOa-Middleware module exposes only one function, greatly improving ease of use; The @Zpfe/KOA-Middleware module itself is more structured internally, more predictable in order of execution, and easier to unit test.

Before declaring refactoring complete, we also need to take a look at the @Zpfe/Koa-Middleware module in its entirety, looking for missing “bad smells” and any that may have accumulated during the gradual refactoring process.

The current @zpfe/ KOa-Middleware module consists of five middleware components, each of which has a registry function that controls its own internal functionality through parameters. The main file of the @zpfe/ KOA-Middleware module is responsible for arranging the parameters passed in by the user into the format expected by each middleware, as follows:

function registerTo(koaApp, options) {
  initiators(koaApp, { configN: options.configN })
  blockers(koaApp, { configO: options.configO })
  preProcessors(koaApp, { configP: options.configP })
  processors(koaApp, { configQ: options.configQ })
  postProcessors(koaApp, { configR: options.configR })
}
Copy the code

Since each middleware needs to get the data it needs from the options parameter of the registration function, it is possible to classify the structure of the options parameter according to the middleware. After classifying the registration function, it will look more concise:

function registerTo(koaApp, options) {
  initiators(koaApp, options.initiators)
  blockers(koaApp, options.blockers)
  preProcessors(koaApp, options.preProcessors)
  processors(koaApp, options.processors)
  postProcessors(koaApp, options.postProcessors)
} 
Copy the code

In the previous analysis, we learned that initializers generate data and want it to be cleaned up by itself, which means that the post-processor has a task to clean up the data. Splitting the initialization and cleanup logic of the same functionality into two files is also a “bad taste.”

The way to handle this is simply to find all the features that have such characteristics and create separate code files for them. It then moves its initialization logic and cleanup logic into the file and exports them, respectively. As a result, each function becomes more cohesive.

The code structure after refactoring is shown below:

conclusion

Looking back at the entire refactoring process, the first thing we did was not code, but take a deep look at the status quo. In this process, some patterns emerge naturally, which are “raw material” for refactoring.

When we actually coded, we took an incremental approach, breaking the process down into multiple steps. Strive to achieve the completion of each step, the entire module can meet the release standard. This means that the changes involved in each step need to be limited to a manageable range, and that each step needs to include complete testing.

So that’s the difference between refactoring and rewriting.

Note: This article was originally published on Zhaopin Big Front Internal Wiki on August 8, 2018.