This article was first published on my personal blogLearn ExpressJs middleware principles
ExpressJs is a popular Node.js Web application framework. After using ExpressJs, I want to learn the source code design of this framework at a deeper level, so as to know why. This article shows how to write a simple MyExpress that mimics the Express implementation to document the source learning of the framework.
Express source code Structure
The Express source code structure looks pretty straightforward.
- Middleware ———— Deals with middleware
- init.js
- query.js
- Router ———— express Information about routes
- index.js
- layer.js
- route.js
- Application. js ———— app module
- Express. js ———— Express application entry file
- Request. js ———— is used to extend the HTTP request
- Response.js ———— is used to extend HTTP response
- Utils. Js ———— utility functions
- View.js ———— Template rendering related
The following will be by imitation Express, to achieve a MyExpress, simplify the learning Express routing module and middleware source code design ideas.
Implement a basic MyExpress
The first step to implementing MyExpress simply involves creating the application and responding to get requests from the client. The functional code to be implemented is as follows:
const express = require("./myexpress");
// Create an express instance
const app = express();
// Process/route requests
app.get("/".(req, res) = > {
res.end("root");
});
// Process /test routing request
app.get("/test".(req, res) = > {
res.end("test");
});
// Listen to port 3000
app.listen(3000.() = > {
console.log("server listening on 3000");
});
Copy the codeAccording to the above requirements, MyExpress needs the following functions:
- MyExpress’s Application constructor, and this constructor needs to provide:
- The LISTEN prototype method is used to listen on a port
- Get prototype method for collecting and responding to GET routes
- Creates a factory function for the MyExpress App instance
As with Express, we create express.js to export a factory function to create an app instance. The code looks like this:
// Express application function (think of it as a constructor)
const Application = require('./application')
function createApplication () {
return new Application()
}
module.exports = createApplication
Copy the codeThe application. Js imported in express.js provides a constructor.
const http = require("http");
const url = require("url");
// Collect routes
const routes = [];
function Application() {}
// Implement the GET route
Application.prototype.get = function (path, handler) {
routes.push({
path,
method: "GET",
handler,
});
};
// Listen and respond
// Internally call the HTTP createServer to get req, res
Application.prototype.listen = function (. args) {
const server = http.createServer((req, res) = > {
const { pathname } = url.parse(req.url);
const method = req.method.toLocaleLowerCase();
// The request path and method are the same, indicating that the route is matched
const route = routes.find(
(route) = >
pathname === route.path && method === route.method.toLocaleLowerCase()
);
if(! route) {// Otherwise respond "404"
return res.end("404");
}
// Responds to the route
route.handler(req, res);
});
// Listen to the portserver.listen(... args); };module.exports = Application;
Copy the codeThe key part of the above code is the LISTEN prototype method. It uses the HTTP module’s createServer method to create an HTTP server instance. This method takes a callback function that takes two arguments, an HTTP Request object and an HTTP Response object.
So you can see from this part of MyExpress implementation that the core of the Express framework is to the HTTP module encapsulation again.
Optimize MyExpress route matching
In the previous step of Application. js, the coupling degree of the logic code that handles the routing response is too high, which is not good for further expansion. According to the source design and implementation of Express, we need to separate the routing function in Application. js and implement an independent routing module.
- Different ways to handle requests.
- Handle matching different request paths.
- Handle dynamic routing path parameters.
To handle the first function point, we need to use the methods third-party library; To handle the second and third function points, we need to use the path-to-Regexp third-party library. These two libraries are also used in Express. Methods provide HTTP methods such as GET, POST, PUT, head, DELETE, options, etc. Path-to-regexp is used to parse routes and convert routes such as /user/:id/:name into regular expressions.
After this step, the application. Js implementation is as follows:
const http = require('http')
const Router = require('./router')
const methods = require('methods')
function App () {
// App routing module
this._router = new Router()
}
methods.forEach(method= > {
// Respond to different HTTP method requests
App.prototype[method] = function (path, handler) {
this._router[method](path, handler)
}
})
App.prototype.listen = function (. args) {
const server = http.createServer((req, res) = > {
// The routing module will handle the routing
this._router.handler(req, res) }) server.listen(... args) }module.exports = App
Copy the codeThe new routing module provides a Router constructor function, which implements many prototype methods corresponding to HTTP methods and a route handler prototype method. The implementation is as follows:
// router/index.js
const url = require('url')
const methods = require('methods')
const pathRegexp = require('path-to-regexp')
function Router () {
// Collect the applied routes
this.stack = []
}
methods.forEach(method= > {
// Collect different HTTP method request routes
Router.prototype[method] = function (path, handler) {
this.stack.push({
path,
method,
handler
})
}
})
Router.prototype.handler = function (req, res) {
const { pathname } = url.parse(req.url)
const method = req.method.toLowerCase()
const route = this.stack.find(route= > {
const keys = []
const regexp = pathRegexp(route.path, keys, {})
// Match the routing path
const match = regexp.exec(pathname)
if (match) {
// Store the dynamic route parameters in the Params object of req
/** Route: /users/:id/name/:name -> /users/123/name/frank Match: ['/users/123/name/frank', '123', 'frank', index: 0, input: '/users/123/name/frank', groups: undefined] key: [{name: 'id', false, offset: 8}, {name: '/users/123/name/frank', groups: undefined] key: [{name: 'id', false, offset: 8}, {name: 'name', optional: false, offset: 29 } ] **/
req.params = req.params || {}
keys.forEach((key, index) = > {
req.params[key.name] = match[index + 1]})}return match && route.method === method
})
// If the route matches, it responds to the route
if (route) {
return route.handler(req, res)
}
res.end('404')}module.exports = Router
Copy the codeAt this stage, MyExpress implementation has touched the core routing function of Express, which is also the essence of Express design. Later, we will realize the function of routing middleware.
Realize MyExpress top-level routing middleware function
The functional code to be implemented is as follows:
const express = require('./express')
const app = express()
app.get('/a'.(req, res, next) = > {
console.log('a 1')
next()
})
app.get('/a'.(req, res, next) = > {
console.log('a 2')
next()
})
app.get('/a'.(req, res, next) = > {
res.end('get /a')
})
app.listen(3000.() = > {
console.log('http://localhost:3000')})Copy the codeAccess http://localhost:3000/a. A 1 A 2 is displayed on the terminal, and GET/A is displayed on the browser
In order to enable next() to call the next route and facilitate the extension of the next parameter function, we need to optimize the implementation of route collection and route match & Handle in router/index.js.
In the Express source code, there is an implementation of the Layer constructor. The Layer is used to service the routing middleware. In this implementation, each Layer instance can be thought of as each of the following routes:
app.get('/a'.(req, res, next) = > {
console.log('a 1')
next()
})
Copy the codeAt this time, the routing module is designed as follows:
Each layer instance carries important information about each route: route path path, route processing handler, dynamic parameter information keys and params obtained by matching, and regular expression regexp of the route. Layer also provides a match prototype method to determine whether the request path matches the route.
Layer. js in the routing module, its implementation is as follows:
// route/layer.js
const pathRegexp = require('path-to-regexp')
function Layer (path, handler) {
this.path = path // Route path
this.handler = handler // Route processing
// Dynamic parameter information
this.keys = []
// The regular expression of the route
this.regexp = pathRegexp(path, this.keys, {})
// Dynamic parameter information
this.params = {}
}
// Used to determine whether the current request route matches
Layer.prototype.match = function (pathname) {
const match = this.regexp.exec(pathname)
if (match) {
this.keys.forEach((key, index) = > {
this.params[key.name] = match[index + 1]})return true
}
return false
}
module.exports = Layer
Copy the codeThe implementation of the modified router/index.js is as follows:
const url = require('url')
const methods = require('methods')
const Layer = require('./layer')
function Router () {
this.stack = []
}
methods.forEach(method= > {
Router.prototype[method] = function (path, handler) {
// Each route corresponds to a layer instance
const layer = new Layer(path, handler)
// Hang in method for matching
layer.method = method
this.stack.push(layer)
}
})
Router.prototype.handler = function (req, res) {
const { pathname } = url.parse(req.url)
const method = req.method.toLowerCase()
// The index of the currently executed route
let index = 0
// Implementation of next
const next = () = > {
// Indicates that all routes have been executed
if (index >= this.stack.length) {
return res.end(`Can not get ${pathname}`)}// Fetch the currently executed route
const layer = this.stack[index++]
// Match the request path
const match = layer.match(pathname)
// Add dynamic request parameters if they match
if (match) {
req.params = req.params || {}
Object.assign(req.params, layer.params)
}
// Execute the route handler
if (match && layer.method === method) {
return layer.handler(req, res, next)
}
// Execute the next route
next()
}
// Enable routing response processing for the application
next()
}
module.exports = Router
Copy the codeImplement multiple middleware functions of MyExpress single routing
On the basis of realizing the function of top-level routing middleware in the previous step, we want to realize the function of multiple middleware corresponding to a single route in this step. The functional code to be implemented is as follows:
const express = require('./express')
const app = express()
// A single route corresponds to multiple processing middleware
app.get('/'.(req, res, next) = > {
console.log('/ 1')
next()
}, (req, res, next) = > {
console.log('/ 2')
next()
}, (req, res, next) = > {
console.log('/ 3')
next()
})
app.get('/'.(req, res, next) = > {
res.end('get /')
})
app.listen(3000.() = > {
console.log('http://localhost:3000')})Copy the codeIn the implementation of the previous step, we introduced the Layer object to optimize the implementation of the routing system. At this step, according to the source implementation of Express, we also need to introduce route (!! The overall routing architecture is implemented as follows:
Router & Route & Layer
- Router The route object is held by the Application
- The Router object collects routing information for the entire application and internally holds a layer in its stack. In this case, the layer holds a route, which can be viewed as a route such as app.get
- The Route Layer holds the route’s final handler function. Each handler is wrapped as a layer instance. All layer instances are stored in the Route stack. So the Route constructor needs to implement a dispatch prototype method that iterates through all the handlers that execute the current routing stack
Router /route.js is implemented as follows (note the comments) :
const methods = require('methods')
const Layer = require('./layer')
function Route () {
// Store the handler layer, that is, each handler
this.stack = []
}
// Iterate over all handler functions in the current routing object
// Out: When all the handlers have been executed, the next app[method] route (route Layer in the figure above) is called.
Route.prototype.dispatch = function (req, res, out) {
// Traverse the inner stack
let index = 0
const method = req.method.toLowerCase()
const next = () = > {
if (index >= this.stack.length) return out()
const layer = this.stack[index++]
if (layer.method === method) {
return layer.handler(req, res, next)
}
next()
}
next()
}
methods.forEach(method= > {
Route.prototype[method] = function (path, handlers) {
handlers.forEach(handler= > {
// Match&Handle is a match&Handle implementation that maps each handler to each handler layer
const layer = new Layer(path, handler)
layer.method = method
this.stack.push(layer)
})
}
})
module.exports = Route
Copy the codeIn order to implement the routing architecture above, we need to modify route/index.js:
- For each app[method], store a layer instance, each such layer holds a Route instance, and each such Route instance holds multiple Handler layers.
- The handle of the routing module needs to implement a chain call to the route of the entire application. Specifically, first call multiple middleware of a single app[method] route, and then call the next single app[method] route.
Route /index.js (!! Note:
const url = require('url')
const methods = require('methods')
const Layer = require('./layer')
const Route = require('./route')
function Router () {
this.stack = []
}
methods.forEach(method= > {
Router.prototype[method] = function (path, handlers) {
const route = new Route()
// The route layer handler is the connection that triggers multiple handlers for the app[method] and the next app[method]
const layer = new Layer(path, route.dispatch.bind(route))
layer.route = route
this.stack.push(layer)
route[method](path, handlers)
}
})
Router.prototype.handle = function (req, res) {
const { pathname } = url.parse(req.url)
let index = 0
const next = () = > {
if (index >= this.stack.length) {
return res.end(`Can not get ${pathname}`)}const layer = this.stack[index++]
const match = layer.match(pathname)
if (match) {
req.params = req.params || {}
Object.assign(req.params, layer.params)
}
// A single request path determines whether the route matches, and the inner handler layer determines the request method
if (match) {
// The handler called here is the dispatch function of the encapsulated route
// The purpose of this is that you can first call multiple middleware for a single app[method] route and then call the next single app[method] route -> next
return layer.handler(req, res, next)
}
next()
}
next()
}
module.exports = Router
Copy the codeAt this point, we complete the core Express routing module
Implement the MyExpress Use method
App.use ([path,] callback [, callback…] )
- The path parameter is optional. The default value is/root path
- Callbacks can be multiple or single
With such a use method, we can “shove” it into the Router module. First, provide the use method in the application:
App.prototype.use = function (path, ... handlers) {
this._router.use(path, handlers)
}
Copy the codeAdd the following to router/index.js:
Router.prototype.use = function (path, handlers) {
// Handle the passed arguments
// If no path is passed
if (typeof path === 'function') {
handlers.unshift(path) // Processing function
path = '/' // Default root path
}
handlers.forEach(handler= > {
// Overlay the design of the app[method]
const layer = new Layer(path, handler)
// tag for special handling
layer.isUseMiddleware = true
this.stack.push(layer)
})
}
Copy the codeFinally, modify the match method in layer.js:
Layer.prototype.match = function (pathname) {
const match = this.regexp.exec(pathname)
if (match) {
this.keys.forEach((key, index) = > {
this.params[key.name] = match[index + 1]})return true
}
// Match the path processing of use middleware
if (this.isUseMiddleware) {
if (this.path === '/') {
return true
}
if (pathname.startsWith(`The ${this.path}/ `)) {
return true}}return false
}
Copy the codeAfter the implementation of the Use method, our MyExpress completed the core design of the simulated Express middleware.
conclusion
After implementing MyExpress, we can learn that the core of Express is the Router module. App is only designed as a top-level structure, providing external use, Listen and multiple HTTP methods. Express middleware has call order and can interrupt calls. The natural slicing of the middleware and the layered design of the routing system really make Express a highly inclusive, fast and minimalist Node.js Web framework.