Use nginx to override URI and proxy functionality in a development environment

Follow me on my blog shymean.com

This article takes a look at ways to override URI and proxy functionality in a development environment using Nginx in front end development.

reference

Nginx Chinese documentation
Essential Nginx knowledge for front-end developers
Nginx and front-end development

The location matching

reference

Nginx location matching

When multiple projects share the same domain name, you need to forward requests to different projects based on the URL. In this case, you need to configure location

location [ = | ~ | ~* | ^~ ] uri { ... }
Copy the code

Optional modifiers can be added between the location directive and the URI request. The four modifiers have the following meanings

= indicates an exact match. A hit is made only if the requested URL path is exactly the same as the following string.
~ indicates that the rule is case sensitive and is defined using re.
~* indicates that the rule is defined using re and is case insensitive.
^~ indicates that if the character following the symbol is the best match, this rule is adopted and no further search is performed.

When no modifiers are added, the request resource path is prefixed with the configured URI: If the request resource path starts with the configured URI, the rule is matched.

Note that the same URI matching rule cannot exist at the same time, i.e

location /img/ {}
location^ ~ /img/ {}
Copy the code

Will prompt an error

nginx: [emerg] duplicate location “/img/” in /usr/local/etc/nginx/servers/test.conf:61

Note that urIs with and without slashes at the end are treated as two matching rules, and they are treated differently, as mentioned in the following example (the article cited above seems to be wrong about this).

The specific matching process of location is

The rules without modifiers are checked first, and prefix matches are performed. The longest match is selected and recorded.
It then checks to see if there is an exact location match (using the = modifier), and if so, ends the search and uses its configuration.
It then looks for an optimal match and, if so, selects the item with the longest optimal match and uses its configuration
The location defined using the regular modifier is then searched sequentially, and if it matches, the search is stopped and the configuration defined by it is used.
Finally, if there is no matching re location, the longest matching prefix character location recorded previously is used.

As can be seen from the above matching process, the matching order is:

Exact match > Optimal match > Sequential regular match > longest prefix match

Next we’ll write some tests to practice location in the following form

Location uri {# config = config [config]}Copy the code

To verify the problem of “which location matches when there are multiple locations”, we can forward the request to a file that does not exist, and then use the error log to see what location the request corresponds to

Now, let’s start testing


server {
    listen 80;
    server_name test.com;
    index index.html;
  
    error_log  /usr/local/etc/nginx/logs/error.log error;
  
    location / {
        root /Users/Txm/A/;
    }
    location /img {
        root /Users/Txm/B/;
    }
    location /img/ {
        root/Users/Txm/C/; }}Copy the code

Next, I prepared some request links so that by visiting and viewing the log, I could see where the request actually went

The request url	Match rule	note
test.com/s/img/1.png	A	Only/matches the prefix matching rule
Test.com/img212/1.pn…	B	Only /img matches the prefix, but /img/ does not
test.com/img/1/1.png	C	There is a difference between /img/ and /img with /
test.com/img/1.png	C	/img/ is a longer and more consistent prefix match than /img

Next, test the re match by adding the following location configuration to the Server module above

location ~* /im {
  root /Users/Txm/D/;
}
location ~* \.png {
  root /Users/Txm/E/;
}
Copy the code

PNG, /img/1/1. PNG, and /img/1. PNG will all match the following rule:

Regular matches have a higher priority than prefix matches, so rule ABC is not matched
The re matches are made in the order defined, and if a match is made, the search is stopped, so rule E is not hit, even though it matches the rule

Let’s go ahead and test ^~ optimal match

location ~ /bund {
  root /Users/Txm/F/;
}
location ~ /bundle/1 {
  root /Users/Txm/F1/;
}
location^ ~ /bundl {
  root /Users/Txm/G/;
}
location^ ~ /bundle/ {
  root /Users/Txm/G1/;
}
location ~ \.js$ {
  root /Users/Txm/H/;
}
Copy the code

We use this link http://test.com/bundle/1.js to test, in theory the link conforms to all of the above rules, actually made the request rule G1, my understanding is:

The priority of optimal match is higher than that of regular match
If multiple optimal matching rules exist, the rule with the longest matching length is matched

Therefore, G1 is matched here. If G1 is deleted, G should be matched according to the priority, and G should be further deleted. At this point, the state reverts to the above re match, and F should be matched according to the rules of re matching in order.

Finally, let’s test exact matching, which means that the requested path and the configured URI are exactly the same

location = /img {
	root /Users/Txm/I/;
}
Copy the code

The request url	Match rule	note
test.com/img	I	Precise matching has the highest priority
test.com/img/	D	It does not satisfy the exact matching and optimal matching, but satisfies the regular matching D in order

Root and alias

reference

Nginx location, root, alias directives

The syntax and matching rules for location are sorted out above, but location does not change the uri of the request. In fact, the requested file path is handled by other directives.

Both root and alias can be used to specify the path to a file. The main difference is how nginx interprets the URI after location, which allows each to map requests to a server file in a different way

Root: root path +location path
Alias: Replace the location path with the alias path

http://test.com/test/index.html request, for example,

server_name test.com;

location /test/ {
  Users/Txm/nginx_test/test/index.html
  Root can be placed under HTTP, server, location, if, etc
  # root /Users/Txm/nginx_test/; 
  
  /Users/Txm/nginx_test/index.html
  # alias can only be placed in a location
  Note that alias must end with a slash
	alias /Users/Txm/nginx_test/; 
}
Copy the code

In other words, alias is the definition of a directory alias, and root is the definition of the topmost directory.

In conjunction with location, use root or Alias to map the requested URL to the corresponding real directory on disk. But at some point, it’s not enough to have a directory without a real file (the most common scenario is probably a development environment that doesn’t generate cached hash values and.min for environment file names). Rewrite can be used to rewrite the request URI path.

rewrite

reference

Rewrite the document
An article about the rewrite module of Nginx

The rewrite module allows you to change the URI of a request using a re, initiate an internal jump to match a location, or simply do a 30X redirect back to the client.

rewrite regex replacement [flag]
Copy the code

Where regex is a PCRE style re, rewrite runs as follows

If the regex matches the URI of the current request, replacement is treated as the new URI for subsequent processing.
- If these options are set at the server level, they will take effect before location.
- If the new URI character contains anything about the protocol, such as http:// or https://, the processing is terminated and the response is 302
If there are multiple rewrite re matching URIs in the same context (server, location, if), the rewrite re will be successively rewritten and replaced according to the order in which the rewrite directive appears, and the replacement will be used as the new URI for subsequent processing
If you want to terminate the match, you can use the third parameter flag, which has the following value
- lastStops processing any rewrite related directives and starts the next round of location matching with the replaced URI
- breakMeans to stop processing any rewrite related directives and use the URI directly to process requests without location matching
- redirectIf there is no protocol and it is a new URI, the request is processed with the location matching the new URI without a 30x jump. But it could just return 30x and let the browser do the redirection itself
- permanentwithredirectThe same, except that it is a direct 301 permanent redirect

Note the difference between last and break. If location is used in location, the redirection will be re-initiated with the new URI and location will be matched again. If both the new URI and the old URI match the same location again, an infinite loop will occur. When this loop is repeated more than 10 times, nginx returns 500. So remember: use last in the server context and break in the location context.

Let’s test the rewrite rule with some examples.

server {
    listen 80;
    server_name test2.com;
    index index.html;
    root  /Users/Txm/nginx_test/;

    access_log  /usr/local/etc/nginx/logs/test2.access.log;
    error_log  /usr/local/etc/nginx/logs/error2.log error;

	  rewrite ^/baidu http://www.baidu.com;
    rewrite ^/bai http://image.baidu.com;

    return 403;
}
Copy the code

The request url	Finally, rewrite’s URI	note
test2.com/bai	image.baidu.com
test2.com/baidu	www.baidu.com	If match to baidu, return directly
And then ADD location

location /bundle/ {
	rewrite^/bundle/(.*?) $ /dist/The $1 break;
}

location /dist/ {
	rewrite^/dist/(.*?) $ /src/The $1 break;
}
Copy the code

The request url	Finally, rewrite’s URI	note
test2.com/bundle/1.js	/Users/Txm/nginx_test/dist/1.js	Break no longer matches the location
test2.com/dist/1.js	/Users/Txm/nginx_test/src/1.js

Then change the break identifier in /bundle/ to last,

location /bundle/ {
  rewrite^/bundle/(.*?) $ /dist/The $1 break;
}
Copy the code

You can see that the final URI, like /dist/, is rewritten to /Users/Txm/nginx_test/ SRC /1.js, so remember to use the break warning in the location.

Rewrite rewriting allows you to rewrite paths to rewrite files that might not otherwise exist on disk. Here is a rewrite rule that strips out the. Min and -hash suffix, mapping static resources packaged using Grunt in historical projects to their SRC counterparts

rewrite ^(.*?) ((\.min)? \ -. *?) (\.. *?) $$1$4 last;
Copy the code

Some uses of the nginx proxy

A reverse proxy serves the server. The reverse proxy helps the server receive requests from clients, forward requests, and balance loads.

The reverse proxy is transparent to the server, but not to us, that is, we do not know that we are accessing a proxy server, and the server knows that the reverse proxy is serving him.

Forward agent is for our service, that is, to serve the client, the client can according to the forward proxy access to itself cannot access to the server resources, an application scenario is: assuming that the company local area network (LAN), are not allowed to access the network, local area network (LAN) of the client to access the Internet, you will need to access through a proxy server.

The forward proxy is transparent to us, but opaque to the server, which does not know whether it is receiving access from the proxy or from the real client.

The following are some of the functional scenarios that can be implemented using nginx agents in front-end development

Simulate online request scenarios in a local development environment

The code running environment can be divided into local development environment, test environment and online production environment. Take an example of a Web project in an existing development process:

The port number is 3015 for local startup at development time
During the test, pull relevant services on the test platform according to the work order, deploy in the container through K8S and run the services, and finally input a series of host lists
When the work order is ready to go live, merge the code into Develop and Master via the publishing platform and deploy it to the production server as instructed

Assume that the link to access the service is http://m.xxx.com/h5/test. In order to achieve the same access scenario in all three environments, the following is generally required:

To access the production environment, enter the current link in the browser. The domain name will be resolved to the IP address of the server. By default, you can access the services in the production environment by entering the domain name.

The test environment can be regarded as a mirror image of the production environment, and the application is deployed on the same server. To access the test environment, you need to set the domain name to the IP address of the test server

During local development, if you need to access the local development environment using a domain name, you can change the host and proxy the domain name to the local Node service

127.0.0.1 xxx.com
Copy the code

Then modify the nginx configuration, using Nginx to proxy the xxx.com domain name request above the local service port number

server {
    listen 80;
    server_name m.xxx.com;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_passhttp://127.0.0.1:3015; }}Copy the code

Here is a super useful host change tool: SwitchHosts, you can easily switch the domain name in the local, test environment, and build environment.

Map the online request resource file locally

Since static resources such as style sheets and JavaScript files in production environments are usually compressed and packaged, and file names are even added with hash suffixes for cache control, there are generally two ways to debug online code at some point

The first method is to use Charles and other packet capture tools to Map the requested file to the Local disk in the mode of Map Local. In this case, the requested resource actually returns the Local file. Then, you can modify the Local file for debugging purposes. This approach is suitable for files that have not been packaged by code merge and can be useful in maintaining older projects that load files using module management tools such as RequireJS and SeaJS.

The second configures the static resource domain name locally, then implements the static resource file proxy through nginx’s location, alias, and rewrite

server  {
    listen 80;
    server_name cnd.shymean.com;
    
    location /wargame/ {
        alias /Users/Txm/github/wargame/dist/;
    		# remove hash a2dfg from request like jquery.min-a2dfg.js link
    		# http://cnd.shymean.com/wargame/jquery.min-a2dfg.js actual return files to/Users/Txm/lot/wargame/dist/jquery. Min. Js
        rewrite ^(.+)/(.+)[-.]\w+\.(\w+)$ The $1/$2.$3 last;
    }
  
    location /blog/ {
    		Map directly to the local directory
        alias/Users/Txm/blog/public/; }}Copy the code

Nginx configuration across domains

A classic scenario of forward proxy is to use Nginx to bypass the cross-domain restriction of the browser. In the development and debugging process of the front and back end, the front-end function of the local may be in the form of localhost:port domain name. In general, the local mock data will be used for development. You will encounter cross-domain problems.

This scenario is mainly caused by the inconsistency between the page domain name (localhost) and interface domain name (api.xxx.com) during development. Through nginx location and proxy_pass, the specified request will be proxy to the corresponding service provider. From the perspective of the browser, the request is the same domain name. There is no cross-domain constraint

server { listen 80; server_name shymean.com; The location/API / {# LAN background in the development of the local service for alignment proxy_pass http://192.168.132.253:7654; }}Copy the code

Here’s another scenario: For the purpose of performance optimization, some static resources often use a separate CDN domain name. When the business needs to use cross-domain resources (such as the need to call Canvas. toDataURL at the end of drawing network pictures on Canvas), there will also be cross-domain restrictions. CORS is very simple to implement with the add_header function of Nginx.

Map $http_origin $imgCorsHost {default 0; "~http://shymean.com" http://shymean.com; "~https://shymean.com" https://shymean.com; } server { listen 80; listen 443; server_name cdn.shymean.com; root /Users/Txm/Desktop/blog/static; add_header Access-Control-Allow-Origin $imgCorsHost; add_header Access-Control-Allow-Methods 'GET, POST, OPTIONS'; add_header Access-Control-Allow-Headers 'DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization'; if ($request_method = 'OPTIONS') { return 204; }}Copy the code

As you can see from the above example, if you need to specify access-Control-Allow-Origin as multiple domain names, you can use nginx’s Map structure.

Modify the content of the response through nginx

Sometimes there is logic that only works in a development environment, such as introducing mock code, adding eruda debugging tools to mobile pages, and so on.

In local development, you can modify the response by specifying the environment variable development at run time to determine if it is a production environment

<% if(! App. IsProduction ()) {% > < script SRC = "https://cdnjs.cloudflare.com/ajax/libs/Mock.js/1.0.1-beta3/mock-min.js" > < / script > <%- IncludeAssets('start:statics/h5/act/js/_mock.js') %> <script> window.isMockReuquest = true </script> <% } %>Copy the code

You can do the same with nginx by intercepting and modifying the response content by adding extra situational code to your code and then injecting mock code. When I first implemented this requirement, I spent a lot of time looking at the implementation methods and realized that the easiest way to do it would be through OpenResty. Reference:

Lua + OpenResty changes the Response body

location ~* 1.js$ {
	body_filter_by_lua_file /usr/local/etc/openresty/lua/hello.lua;
}
Copy the code

Then add the hello.lua script and write the related logic

-- body_filter_by_lua, body filter module, ngx.arg[1] represents the chunk input, and ngx.arg[2] represents whether the current chunk is last
local chunk, eof = ngx.arg[1], ngx.arg[2]
local buffered = ngx.ctx.buffered
if not buffered then
   buffered = {}  -- XXX we can use table.new here 
   ngx.ctx.buffered = buffered
end
if chunk ~= "" then
   buffered[#buffered + 1] = chunk
   ngx.arg[1] = nil
end
if eof then
   local whole = table.concat(buffered)
   ngx.ctx.buffered = nil
	 whole = string.gsub(whole, "console.log"."console.warn")

   ngx.arg[1] = whole
end
Copy the code

Note that the length of the whole content cannot exceed the original length, otherwise the following data will be truncated, presumably related to the content-Length header. I haven’t heard much about OpenResty and Lua before, but I’ll learn more about them later.

summary

This article summarizes several directives related to nginx matching URIs, including

uselocationMatching the uri
userootandaliasSpecify the requested resource directory
userewriteRewrite the URI and subsequent matching rules

By combining URIs and proxies, we can direct requests to the resources we need to meet the multiple development requirements of our development environment.

Use nginx to override URI and proxy functionality in a development environment

The location matching

Root and alias

rewrite

Some uses of the nginx proxy

Simulate online request scenarios in a local development environment

Map the online request resource file locally

Nginx configuration across domains

Modify the content of the response through nginx

summary

Related Posts

Module build failed: Error: ENOENT: no such file or directory, scandir ‘D:\… \n

The front-end qiankun

Disallow web page scaling solutions