In the last article, we created a Web server with basic static file service functionality, but without dynamic functionality. We hope to further expand this program into a fully functional dynamic Web server. But before you get too busy writing code, let’s think about what a Web framework should look like at an architectural level.

Web processing pipeline

While different programming languages implement Web servers in different ways, take a look around and you’ll see that most of the implementations are just details and the core ideas are pretty much the same. The key to this core idea is this:

The Web framework is a processing pipeline.

What does that mean? If you think about what a Web server does, you’ll see that certain tasks are necessary for almost any type of server and are handled almost invariably:

  • Logging;
  • IP filtering;
  • Static file support;
  • The cache.
  • HTTP compression;
  • HTTP protocol parsing;
  • .

Some other features, though required by the server, are handled in very different detail depending on how the framework is implemented:

  • The Session;
  • User authentication and authorization;
  • View engine;
  • Routing;
  • .

We want to decouple the Design of the Web server itself, and the functionality that the Web server is supposed to support, allowing them to vary independently. Such a design opens up many possibilities:

  • Each function can be broken down into small, independent components that can be deployed, tested, and reused independently;
  • The Web server can flexibly enable or disable various functions in a plugin-like mechanism under the premise that the core is stable, so as to maintain a good balance between functions and performance.
  • Allow multiple applications to be hosted in a single Web service (most modern Web servers have plug-ins that support multiple languages);
  • Allow multiple Web servers to be connected, with each browser hosting what it does best (Apache/Nginx as a front-end reverse proxy, and dynamic servers on the back end being the most common pattern);
  • At the application level, processing steps can be added, modified, or deleted dynamically through configuration or code to achieve deep customization of the process.

This design idea is so good that most popular Web servers today are designed along these lines. Interested people can read the Wikipedia entry HTTP Pipeline for many resources and links. Of course, there are many differences between languages and frameworks in the specific implementation methods. For example, JavaEE architectures generally use filters or custom servlets. ASP.NET is divided into HttpFilter and HttpModule, the subsequent ASP.NET MVC provides more extension points; The Connect/Express architecture that is widely used in Nodejs is called Middleware in general. Our example here is also called Middleware, as this seems to be the unwritten convention of most frameworks these days.

Note: The original book was written in 2015, and perhaps due to its early years (or perhaps due to the author’s personal style), the book calls each step WorkflowItem, rather than adopting the more popular term among popular Web frameworks. I prefer the Middleware term myself and write it differently from the original. I don’t think my code is necessarily better than the original book; If you prefer the original author’s style, please download and read for yourself.

code

The sample code of this article has been put on Github, and the code associated with each article is placed in a separate branch for readers’ reference. Therefore, to get the sample code for this article, use the following command:

git clone https://github.com/shuhari/web-server-succinctly-example.git
git checkout -b  02-middlewares origin/02-middlewares 
Copy the code

implementation

Implementing a Web processing pipeline is not difficult: all you need to do is create a queue of Middleware and call them in turn. Of course, some middleware is transitive in nature (processing is done and the next person takes over); Others are abortive (they’ve already done their job, so there’s no need for anyone else to step in), and our code needs to handle these cases properly. Also, given that middleware execution may throw exceptions, it is best to create a global exception handling hook to keep the server from crashing — Nodejs treats it as middleware, but with an extra exception parameter to pass. Given the nature of the C# language, we’ll declare it as an interface. Start by defining what the middleware might return:

public enum MiddlewareResult
{
    Processed = 1,
    Continue = 2,
}
Copy the code

The code on Github includes comments, but there are text explanations in the article, and comments have been removed from the code for brevity.

Next, define the interface for the middleware (and the error-handling hooks) :

public interface IMiddleware
{
    MiddlewareResult Execute(HttpListenerContext context);
}

public interface IExceptionHandler
{
    void HandleException(HttpListenerContext context, Exception exp);
}
Copy the code

Define a MiddlewarePipeline class, which registers and executes each middleware in turn:

class MiddlewarePipeline { public MiddlewarePipeline() { _middlewares = new List<IMiddleware>(); } private readonly List<IMiddleware> _middlewares; private IExceptionHandler _exeptionHandler; internal void Add(IMiddleware middleware) { _middlewares.Add(middleware); } internal void UnhandledException(IExceptionHandler handler) { _exeptionHandler = handler; } internal void Execute(HttpListenerContext context) { try { foreach (var middleware in _middlewares) { var result = middleware.Execute(context); if (result == MiddlewareResult.Processed) { break; } else if (result == MiddlewareResult.Continue) { continue; } } } catch (Exception ex) { if (_exeptionHandler ! = null) _exeptionHandler.HandleException(context, ex); else throw; }}}Copy the code

To allow the application to customize the execution steps of the middleware, we define a configuration interface that allows the application to decide which middleware to use:

public interface IWebServerBuilder
{
    IWebServerBuilder Use(IMiddleware middleware);

    IWebServerBuilder UnhandledException(IExceptionHandler handler);
}
Copy the code

Recall that the current WebServer class handles static files by itself, and now we can delegate the work to the WorkflowPipeline implemented above, making the work of the WebServer itself much easier (only the changes are listed here to avoid missing the point) :

public class WebServer : IWebServerBuilder { ... private readonly MiddlewarePipeline _pipeline; public WebServer(int concurrentCount) { ... _pipeline = new MiddlewarePipeline(); } public void Start() { _listener.Start(); Task.Run(async () => { while (true) { _sem.WaitOne(); var context = await _listener.GetContextAsync(); _sem.Release(); _pipeline.Execute(context); }}); } public IWebServerBuilder Use(IMiddleware middleware) { _pipeline.Add(middleware); return this; } public IWebServerBuilder UnhandledException(IExceptionHandler handler) { _pipeline.UnhandledException(handler); return this; }}Copy the code

Now that everything is in place, we can write some middleware to validate the architecture. Following the common structure of a general Web application, our sample application adds the following functional middleware:

  • Logging HTTP requests (simulated by Console);
  • Allow blocking requests by IP address (blacklist);
  • Provide static files;
  • If none of the middleware responds, HTTP 404 is returned;
  • Finally, if there is a middleware execution error, HTTP 500 is returned.

The program entry class should look like this:

internal class Program { ... public static void Main(string[] args) { var server = new WebServer(concurrentCount); RegisterMiddlewares(server); . } static void RegisterMiddlewares(IWebServerBuilder builder) { builder.Use(new HttpLog()); / / builder. Use (new BlockIp (" : "1", "127.0.0.1")); builder.Use(new StaticFile()); builder.Use(new Http404()); builder.UnhandledException(new Http500()); }}Copy the code

For example purposes, our middleware is as simple as implementing a method of the IMiddleware interface. However, we found that all HTTP errors are handled similarly, so we implemented a common helper method first;

public static class HttpUtil { public static HttpListenerResponse Status(this HttpListenerResponse response, int statusCode, string description) { var messageBytes = Encoding.UTF8.GetBytes(description); response.StatusCode = statusCode; response.StatusDescription = description; response.ContentLength64 = messageBytes.Length; response.OutputStream.Write(messageBytes, 0, messageBytes.Length); response.OutputStream.Close(); return response; }}Copy the code

Finally, we look at the implementation of each middleware. HttpLog outputs input information to the console. Real applications need to be configured to log, but we’re here to illustrate how this works. But as you can see, this class is already decoupled from the rest of the program, so it’s not difficult to extend it to support logging without worrying about compromising other functionality — which is the strength of the Middleware architecture.

public class HttpLog : IMiddleware { public MiddlewareResult Execute(HttpListenerContext context) { var request = context.Request; var path = request.Url.LocalPath; var clientIp = request.RemoteEndPoint.Address; var method = request.HttpMethod; Console.WriteLine("[{0:yyyy-MM-dd HH:mm:ss}] {1} {2} {3}", DateTime.Now, clientIp, method, path); return MiddlewareResult.Continue; }}Copy the code

BlockIp implements blacklist-like functions. This is a useful feature if you find that an IP address is an attacker, or if you simply don’t want him or her to see your site – although in a production environment it is possible to implement this feature at the reverse proxy level or further forward for better performance.

public class BlockIp : IMiddleware { public BlockIp(params string[] forbiddens) { _forbiddens = forbiddens; } private string[] _forbiddens; public MiddlewareResult Execute(HttpListenerContext context) { var clientIp = context.Request.RemoteEndPoint.Address; if (_forbiddens.Contains(clientIp.ToString())) { context.Response.Status(403, "Forbidden"); return MiddlewareResult.Processed; } return MiddlewareResult.Continue; }}Copy the code

StaticFile is basically the StaticFile handling part of the original WebServer. Of course, this implementation is certainly problematic for file types other than.html. But the search for file types is tedious and not very technical, so I won’t expand here. The class can also be easily extended to support other file types.

public class BlockIp : IMiddleware { public BlockIp(params string[] forbiddens) { _forbiddens = forbiddens; } private string[] _forbiddens; public MiddlewareResult Execute(HttpListenerContext context) { var clientIp = context.Request.RemoteEndPoint.Address; if (_forbiddens.Contains(clientIp.ToString())) { context.Response.Status(403, "Forbidden"); return MiddlewareResult.Processed; } return MiddlewareResult.Continue; }}Copy the code

Error handling is also straightforward with the helper methods written above:

public class Http404 : IMiddleware { public MiddlewareResult Execute(HttpListenerContext context) { context.Response.Status(404, "File Not Found"); return MiddlewareResult.Processed; } } public class Http500 : IExceptionHandler { public void HandleException(HttpListenerContext context, Exception exp) { Console.WriteLine(exp.Message); Console.WriteLine(exp.StackTrace); context.Response.Status(500, "Internal Server Error"); }}Copy the code

Again, the code above is intended to illustrate the implementation and is certainly not as robust as production code — but I don’t want too much error handling to obscure the focus of the article. If you need to implement a production-level server, the above processing code needs to be carefully designed to support a variety of possible scenarios. However, we have designed a flexible system architecture so that users of the framework can easily add a variety of custom functions.

Of course, our code at this point does not have the capability to support dynamic services. This is also the subject of our next article: routing.

series

  • Write your own Web server (index) in C#
  • Write a Web server in C#, part 1 – basics
  • Write a Web server in C#, part 2 – middleware
  • Write a Web server in C#, part 3 – routing
  • Write your own Web server in C#, part 4 – Session
  • Write your own Web server in C#, part 5 – view engine
  • Writing your own Web server in C#, part 6 – user authentication