Building wheels is one of the best ways to learn. This article attempts to build a Python Web framework wheel from 0, which I call ToyWebF.

Disclaimer: This article refers to Jahongir Rahmonov’s How to Write a Python Web Framework. Series of articles written, content structure will not be the same, please rest assured to eat.

The operating environment of this document is MacOS. Replace the commands in this document based on your own operating system.

ToyWebF’s simple features:

  • 1. Supports various route registration modes
  • 2. Supports static HTML, CSS, and JavaScript
  • 3. Supports custom errors
  • 4. Support middleware

Let’s implement these features.

The simplest Web service

First, we need to install Gunicorn. Recall the Flask framework, which has a built-in Web server, but is unstable, so when it goes live it is usually replaced by uWSGI or Gunicorn. Instead of using the built-in Web service, we use Gunicorn.

We create a new directory with the Python virtual environment in which gunicorn is installed

mkdir ToyWebF
python3 -m venv venv Create a virtual environment
source venv/bin/activate Activate the virtual environment
pip install gunicorn
Copy the code

To build the simplest Web service with nothing, create app.py and api.py files in ToyWebF and write the following code.

# API. Py files

class API:
    def __call__(self, environ, start_response):
        response_body = b"Hello, World!"
 status = "200 OK"  start_response(status, headers=[])  return iter([response_body])  # app. Py files from api import API  app = API() Copy the code

Run the Gunicorn app: visit http://127.0.0.1:8000 and you can see Hello, World! However, the parameters in the request body are difficult to parse in environ variables, and the response returned is also bytes.

We can use the WEBob library to convert environ data to Request and return data to Response.

pip install webob
Copy the code

Then modify the API class’s __call__ method as follows.

from webob import Request, Response

class API(object):
    def wsgi_app(self, environ, start_response):
        Convert the requested environment information into a Request object using weBob
 request = Request(environ)  response = self.handle_request(request)  return response(environ, start_response)   def __call__(self, environ, start_response):  self.wsgi_app(environ, start_response) Copy the code

The Request class of the WEBob library converts environ into an easily processed Request. The handle_REQUEST method is called to process the Request. The response object returns the processing result.

Handle_request method is very important in ToyWebF. It will match the processing method corresponding to a route, and then call this method to process the request and return the processing result. Before parsing HANDle_Request, we need to discuss the implementation of route registration, as shown in the following code.

class API(object):

    def __init__(self):
        # url routing
        self.routes = {}
  def route(self, path):  Add route decorator  def wrapper(handler):  self.add_route(path, handler)  return handler  return wrapper   def add_route(self, path, handler):  The same path cannot be added repeatedly  assert path not in self.routes, "Such route already exists"  self.routes[path] = handler Copy the code

Add routes and methods to the self.routes dictionary. You can associate a route with a route decorator or use the add_route method in app.py.

app = API()

Use decorators to correlate routes and methods
@app.route("/home")
def home(request, response):
 response.text = "This is Home"  Routes can have variables, and corresponding methods need to have corresponding parameters @app.route("/hello/{name}") def hello(requst, response, name):  response.text = f"Hello, {name}"  Can decorate the class @app.route("/book") class BooksResource(object):  def get(self, req, resp):  resp.text = "Books Page"  def handler1(req, resp):  resp.text = "handler1" # this can be added directly through the add_route method app.add_route("/handler1", handler1) Copy the code

The url can contain variables, such as @app.route(“/hello/{name}”), so it needs to be parsed when matching. We can use the regular matching method to match.

# pip install parse

In [1] :from parse import parse

# matches
In [2]: res = parse("/hello/{name}"."/ hello/two two")  In [3]: res.named Out[3] : {'name': 'the two'} Copy the code

The find_handler method is defined to iterate over self.routes.


class API(object):

    def find_handler(self, request_path):
        # Route traversal
 for path, handler in self.routes.items():  # rematch routes  parse_result = parse(path, request_path)  if parse_result is not None:  Return the route method and route itself  return handler, parse_result.named  return None.None Copy the code

After understanding the principle of route and method association, handle_REQUEST method can be implemented. The main path of this method is the corresponding method based on route scheduling. The code is as follows.

import inspect
class API(object):

    def handle_request(self, request):
        """ Request scheduling """
  response = Response()  handler, kwargs = self.find_handler(request.path)   try:  if handler is not None:  if inspect.isclass(handler): If it is a class, get the method in it  handler = getattr(handler(), request.method.lower(), None)  if handler is None: If the method does not exist in the class, the request type is not supported by the class  raise AttributeError("Method now allowed", request.method)  handler(request, response, **kwargs)  else:  # return default error  self.defalut_response(response)  except Exception as e:  raise e  return response Copy the code

In this method, the Response object of weBob library is instantiated first, and then the method and parameters corresponding to the request route are obtained through self.find_handler method, such as.

@app.route("/hello/{name}")
def hello(requst, response, name):
    response.text = f"Hello, {name}"
Copy the code

It will return the Hello method object and the name argument, or if it is /hello/ two-two, then name is two-two.

Because the Route decorator may decorate the class object of the route decorator, for example.

Can decorate the class
@app.route("/book")
class BooksResource(object):
    def get(self, req, resp):
        resp.text = "Books Page"
Copy the code

The hanler returned by self.find_handler is a class, but we want to call the get, post, and delete methods of the class, so we need a simple inspection logic to determine if the handler is a class object. The getattr method is used to get the corresponding request method in the instance of the class object.

Request.method.lower () can be used for get, post, or delete
handler = getattr(handler(), request.method.lower(), None)
Copy the code

If the method attribute does not exist in the class object, an error is thrown that the request type is not allowed. If the method attribute is not in the class object or exists in the class object, the call directly can.

In addition, if the method’s route is not registered with self.routes, as in 404, the defalut_response method is defined to return it, as follows.

class API(object):

    def defalut_response(self, response):
        response.status_code = 404
        response.text = "Not Found"
Copy the code

If the process scheduled in the HANDle_REQUEST method is faulty, raise directly raises the error.

At this point, one of the simplest Web services has been written.

Static file support

Flask can support static files like HTML, CSS, JavaScript, etc. Using template language, we can build simple but beautiful Web applications. We let TopWebF support this function, and finally realize the website in the picture, perfectly compatible with static files.


Flask uses Jinja2 as its HTML template engine, as does ToyWebF. Jinja2 is actually a simple DSL (domain language) that allows us to change the structure of HTML with special syntax, and it’s a project worth studying.

PIP install Jinja2, then you can use it, and create the templates directory in the ToyWebF project directory as the default ROOT for your HTML file.

from jinja2 import Environment, FileSystemLoader

class API(object):

    def __init__(self, templates_dir="templates"):
 # HTML folder  self.templates_env = Environment(loader=FileSystemLoader(os.path.abspath(self.templates_dir)))   def template(self, template_name, context=None):  """ Return template content """  if context is None:  context = {}  return self.templates_env.get_template(template_name).render(**context) Copy the code

First, use jinja2 FileSystemLoader class to use a folder in File system as loader, and then initialize the Environment.

In the process of use (that is, calling the template method), the get_template method is used to get a specific template and the render method is used to pass the corresponding content to the variables in the template.

Here we don’t write front-end code, go directly to the Internet to download the template, here to download the Bootstrap provide free template, can go to https://startbootstrap.com/themes/freelancer/ to download, after the download, You’ll get index.html and the corresponding CSS, JSS, img files, etc. Move the index.html to ToyWebF/ Templates and make some simple changes, adding some variables.

<! -- Masthead Heading-->
<h1 class="masthead-heading text-uppercase mb-0">{{ title }}</h1>
<! -- Masthead Subheading-->
<p class="masthead-subheading font-weight-light mb-0">-{{name}}</p>
Copy the code

You then define the route and the required parameters for index.html in the app.py file.

@app.route("/index")
def index(req, resp):
    template =  app.template("index.html", context={"name": "Two"."title": "ToyWebF"})
    # resp.body requires bytes. The template method returns a Unicode string, so encoding is required
    resp.body = template.encode()
Copy the code

Support for HTML files is complete at this point, but the HTML does not load CSS and JS properly, resulting in ugly page layouts and unusable interactions.

ToyWebF supports CSS, JS and img files. Create a static folder in ToyWebF to store static files such as CSS, JS and img files.

Using the Whitenoise third-party library, you can make your Web framework support CSS and JS with a few simple lines of code, without relying on nginx and other services. First, PIP install Whitenoise, and then modify the __init__ method of the API class.

class API(object):

    def __init__(self, templates_dir="templates", static_dir="static"):
        # HTML folder
        self.templates_env = Environment(loader=FileSystemLoader(os.path.abspath(self.templates_dir)))
 # CSS, JavaScript folder  self.whitenoise = WhiteNoise(self.wsgi_app, root=static_dir) Copy the code

This is done by wrapping the self.wsgi_app method in WhiteNoise and calling self. WhiteNoise directly when calling the API’s __call__ method.

class API(object):
    def __call__(self, environ, start_response):
        return self.whitenoise(environ, start_response)
Copy the code

In this case, if a Web service is requested to obtain static resources such as CSS and JS, WhiteNoise will obtain the content and return it to the client. WhiteNoise will match the corresponding files of the static resources in the system and read them back.

At this point, the beginning of the web page effect is achieved.

Custom error

The Web service returns an Internal Server error by default if 500 is present, which is ugly, and some code needs to be added so that the framework user can customize the error returned at 500.

First when the API initializes, it initializes the self.Exception_handler object and defines the corresponding method to add the custom error

class API(object):

    def __init__(self, templates_dir="templates", static_dir="static"):
        # Custom error
        self.exception_handler = None
  def add_exception_handler(self, exception_handler):  Add a custom error handler  self.exception_handler = exception_handler Copy the code

When the handler_request method performs request scheduling, the scheduled method executes logic times 500. Instead of throwing an error by default, the scheduled method determines whether there is custom error handling.

class API(object):

    def handle_request(self, request):
        """ Request scheduling """

 try:  #... omit  except Exception as e:  An internal server error is returned if # is null  if self.exception_handler is None:  raise e  else:  # Custom error return form  self.exception_handler(request, response, e)  return response Copy the code

In app.py, customize the error return method as follows.


def custom_exception_handler(request, response, exception_cls):
    response.text = "Oops! Something went wrong."

# Custom error
app.add_exception_handler(custom_exception_handler) Copy the code

The custom_Exception_handler method only returns a custom paragraph, which you can easily replace with a nice template.

We can experimentally define a route to see the effect.

@app.route("/error")
def exception_throwing_handler(request, response):
    raise AssertionError("This handler should not be user")
Copy the code

Middleware support

The middleware of Web services can also be understood as hooks, that is, to do something with the request before it is requested or to do something with the Response before it is returned.

To support middleware, create the middleware.py file in the TopWebF directory.

Review the scheduling logic of the request now.

1. Use the Routes decorator to associate routes and methods. 2. Wsgi_app will eventually call the api.handle_REQUEST method to get the method corresponding to the route and call it to execute the corresponding logic

If you want to do something before and after request, you need to run the logic before and after APi.handle_REQUEST.

from webob import Request

class Middleware(object):
    def __init__(self, app):
        self.app = app # API class instance
  def add(self, middleware_cls):  # Instantiate the Middleware object and wrap around self.app  self.app = middleware_cls(self.app)   def process_request(self, req):  # request  pass   def process_response(self, req, resp):  # Response the processing to be done after  pass   def handle_request(self, request):  self.process_request(request)  response = self.app.handle_request(request)  self.process_response(request, response)  return response   def __call__(self, environ, start_response):  request = Request(environ)  response = self.app.handle_request(request)  return response(environ, start_response) Copy the code

The Add method instantiates the Middleware object, which wraps around the current API class instance.

The middleware.handle_REQUEST method is called self.process_request before self.app.handle_REQUEST to handle the data before the request and self.process_response Data after response, and the core scheduling logic is still processed by apI. handle_REQUEST method.

The __call__ and handle_request methods both have self.app.handle_request(request), but the objects they call seem to be different. I’ll leave that for now, work on the code, and then come back.

You then create the Middleware properties for the API in api.py along with methods to add new middleware.

class API(object):

    def __init__(self, templates_dir="templates", static_dir="static"):
        Request middleware, pass in the API object
        self.middleware = Middleware(self)
  def add_middleware(self, middleware_cls):  # Add middleware  self.middleware.add(middleware_cls) Copy the code

Then, in app.py, you customize a simple piece of middleware and add it by calling the add_middleware method.

class SimpleCustomMiddleware(Middleware):
    def process_request(self, req):
        print("Processing request", req.url)

    def process_response(self, req, resp):
 print("Handling the response", req.url)  app.add_middleware(SimpleCustomMiddleware) Copy the code

After the definition of middleware, in the request scheduling, it is necessary to use middleware, in order to be compatible with static files, CSS, JS, ING file request path to do compatibility, add /static prefix in its path

<! Change the path to import static files in index. HTML by simply adding the /static prefix. -->

<! Static static static static static static static static static static static static static static
<link href="/static/css/styles.css" rel="stylesheet" />
<img class="masthead-avatar mb-5" src="/static/assets/img/avataaars.svg" alt="" />
<script src="/static/js/scripts.js"></script> Copy the code

Next, change the API’s __call__ to accommodate middleware and static files as follows.

class API(object):

    def __call__(self, environ, start_response):
        path_info = environ["PATH_INFO"]
        static = "/" + self.static_dir
 # start with /static or middleware is empty  if path_info.startswith(static) or not self.middleware:  # "/static/index.css" -> select /index.css  environ["PATH_INFO"] = path_info[len(static):]  return self.whitenoise(environ, start_response)  return self.middleware(environ, start_response) Copy the code

At this point, the middleware logic is complete.

The Middleware class’s __call__ and handle_request methods call self.app.

For easy understanding, here is a step by step breakdown.

If no new middleware is added, the scheduling logic for the request is as follows.

# Attribute mapping
API.middleware = Middleware
API.middleware.app = API

# scheduling logic
API.__call__ -> middleware.__call__ -> self.app.handle_request -> API.handle_request() Copy the code

Without middleware, self.app is the API itself, so self.app. Handle_request in middleware.__call__ calls API.handle_Request.

If a new middleware is added, such as one called SimpleCustomMiddleware in the code above, the request scheduling logic is as follows.

# Attribute mapping
API.middleware = Middleware
API.middleware.app = API
API.middleware.add(SimpleCustomMiddleware)
API.middleware.app = SimpleCustomMiddleware
API. The middleware. The app. The app = API equivalent of the API. Middleware. SimpleCustomMiddleware. App = API # scheduling logic API.__call__ -> middleware.__call__ -> self.app.handle_request -> SimpleCustomMiddleware.handle_request() -> self.app.handle_request -> API.handle_request() Copy the code

Because when registering Middleware, the middleware.add method replaces the app object in the original Middleware instance with SimpleCustomMiddleware, which also has app objects, The App object in SimpleCustomMiddleware is an instance of an API class.

During request scheduling, the Middleware class’s handle_REQUEST method is triggered, which executes Middleware logic to process the data in request and Response.

Of course, you can add more than one piece of Middleware through the middleware.add method, which creates a stack call effect, as shown below.

class SimpleCustomMiddleware(Middleware):
    def process_request(self, req):
        print("Processing request", req.url)

    def process_response(self, req, resp):
 print("Handling the response", req.url)   class SimpleCustomMiddleware2(Middleware):  def process_request(self, req):  print("Handling request2", req.url)   def process_response(self, req, resp):  print("Handling response2", req.url)  app.add_middleware(SimpleCustomMiddleware) app.add_middleware(SimpleCustomMiddleware2) Copy the code

Once the Web service is started, its execution results are as follows.


The tail

I have been learning about the principles of compilation myself, and I will use this knowledge to create my own simple language and share the process in the form of an article.

I’ll see you in the next article. Oh, and remember to “watch” because no one else is watching.