Cache is an important part of improving application performance. Write an article summarizing the various caches used for dynamic content.
The article uses Nginx, Rails, Mysql, Redis as examples, and other Web servers, languages, databases, and caching services are similar.
The following is a schematic diagram of the three layers for subsequent reference:
1. Client cache
A client will often access the same resource, such as the homepage of a website or the same article in a browser, or the same API in an app. If the resource is unchanged from the previous one, You can use the HTTP specification of 304 Not Modified response headers (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5), directly with the client’s cache, Without having to regenerate content once on the server side.
Rails has a built-in fresh_when method that can be done in one line of code:
class ArticlesController
def show
@article = Article.find(params[:id])
fresh_when :last_modified => @article.updated_at.utc, :etag => @article
endend
The next time the user accesses the request header, if-modified-since and if-none-match will be compared. If they Match, 304 will be returned instead of generating the Response body.
However, this will encounter a problem, suppose that our website navigation has user information, a user in the not logged in topic to visit, and then logged in later to visit, will find that the page displayed is still not logged in status. Or you can visit an article in the APP and bookmark it. The next time you enter the article, the state of unbookmark will still be displayed. The solution to this problem is simply to add user-specific variables to the eTAG calculation:
fresh_when :etag => [@article.cache_key, current_user.id]
fresh_when :etag => [@article.cache_key, current_user_favorited]
In addition, if Nginx enables gzip to compress the result of rails execution, it will dry out the etag header of Rails output. The developers of Nginx say that according to the RFC specification, proxy_pass must be handled this way (because the content changes). SRC/HTTP /modules/ngx_http_gzip_filter_module.c: SRC/HTTP /modules/ngx_http_gzip_filter_module.c: SRC/HTTP /modules/ngx_http_gzip_filter_module.
//ngx_http_clear_etag(r);
Or you can choose not to change the nginx source code, take gzip off, and use Rack middleware to handle compression:
config.middleware.use Rack::Deflater
In addition to specifying fresh_WHEN in the controller, the Rails framework defaults to using Rack::ETag Middleware, which automatically adds an ETag to a non-Etag response, but compared to fresh_WHEN, The automatic eTAG saves the client time. The server will still execute all the code.
Rack::ETag Automatically adds to ETag:
curl -v http://localhost:3000/articles/1 < Etag: "bf328447bcb2b8706193a50962035619" < X-Runtime: 0.286958 the curl -v http://localhost:3000/articles/1 -- header 'If - None - Match: "Bf328447bcb2b8706193a50962035619" '< X - the Runtime: 0.293798
Use fresh_when:
curl -v http://localhost:3000/articles/1 --header 'If-None-Match: "bf328447bcb2b8706193a50962035619"' < X-Runtime: 0.033884
2. Nginx cache
Some resources, such as list apis on news apps and Ajax request categorization menus on shopping websites, may be called a lot, but are not related to the user’s state and rarely change. Consider using Nginx for caching.
There are two main implementation methods:
A. Dynamic request static file
After a Rails request completes, the result is saved to a static file, and subsequent requests are supplied directly by nginx, using after_filter:
class CategoriesController < ActionController::Base
after_filter :generate_static_file, :only => [:index]
def index
@categories = Category.all
end
def generate_static_file
File.open(Rails.root.join('public', 'categories'), 'w') do |f|
f.write response.body
end
endend
In addition, we need to delete this file when any classification update, to avoid cache does not refresh the problem:
class Category < ActiveRecord::Base
after_save :delete_static_file
after_destroy :delete_static_file
def delete_static_file
File.delete Rails.root.join('public', 'categories')
endend
Prior to Rails 4, this generated static file caching could be handled with the built-in caches_page. After Rails 4, it became a separate gem actionpack-page_Caching, compared to manual code.
class CategoriesController < ActionController::Base
caches_page :index
def update
#...
expire_page action: 'index'
endend
If there is only one server, this method is simple and practical, but if there are multiple servers, there will be a problem that the update category can only refresh its own server cache. You can use NFS to share static resource directory to solve the problem, or use the second method:
B. Statically implement the centralized cache service
First we need to give Nginx the ability to access the cache directly:
upstream redis {
server redis_server_ip:6379;
}
upstream ruby_backend {
server unicorn_server_ip1 fail_timeout=0;
server unicorn_server_ip2 fail_timeout=0;
}
location /categories {
set $redis_key $uri;
default_type text/html;
redis_pass redis;
error_page 404 = @httpapp;
}
location @httpapp {
proxy_pass http://ruby_backend;
}
Nginx first fetchs the requested URI as a key from redis. If it doesn’t get (404), it forwards the request to unicorn for processing. Then overwrite generate_static_file and delete_static_file methods:
redis_cache.set('categories', response.body)
redis_cache.del('categories')
In addition to centralized management, it can also set the expiration time of cache. For some data with no timeliness requirements, it can simply refresh at a fixed time without dealing with the refresh mechanism:
redis_cache.setex('categories', 3.hours.to_i, response.body)
3. Full page caching
Nginx caching can be very difficult to handle when processing requests with parameter resources or user state, where full-page caching can be used.
For example, in a list of paging requests, we can add the page parameter to cache_path:
class CategoriesController
caches_action :index, :expires_in => 1.day, :cache_path => proc {"categories/index/#{params[:page].to_i}"}end
For example, we only need to cache the RSS output for 8 hours:
class ArticlesController caches_action :index, :expires_in => 8.hours, :if => proc {request.format.rss? }end
For example, for non-logged-in users, we can cache the home page:
class HomeController caches_action :index, :expires_in => 3.hours, :if => proc {! user_signed_in? }end
4. Fragment caching
While the first two caches have limited scenarios, fragment caches are the most widely applicable.
Scenario 1: We need a piece of AD code for each page to display different ads. If fragment caching is not used, each page will query the AD code and take some time to generate the HTML code:
- if advert = Advert.where(:name => request.controller_name + request.action_name, :enable => true).first
div.ad
= advert.content
With the addition of fragment caching, this query can be eliminated:
- cache "adverts/#{request.controller_name}/#{request.action_name}", :expires_in => 1.day do
- if advert = Advert.where(:name => request.controller_name + request.action_name, :enable => true).first
div.ad
= advert.content
Scenario 2: Read the article, the content of the article may not change for a long time, often changes may be article comments, you can add fragment cache to the body of the article:
- cache "articles/#{@article.id}/#{@article.updated_at.to_i}" do
div.article
= @article.content.markdown2html
Activerecord’s cache_key method uses updated_at as its default cache_key method. You can also add more parameters, such as a counter cache that has comments on an article. Updating the number of comments does not update the time of the article. You can add this counter to the key as well
Scenario 3: Generation of complex page structures
The page data structure is more complex, avoid when generating a large number of queries and HTML rendering, use fragment caching, this part can be greatly save time, to travel notes page on our website http://chanyouji.com/trips/109123 (please allow a little a advertising, with some traffic) :
You need to get weather data, photo data, text data, etc., and generate META, keyword and other SEO data, which are intercrossed with other dynamic content. The fragment cache can be divided into multiple:
- cache "trips/show/seo/#{@trip.fragment_cache_key}", :expires_in => 1.day do
title #{trip_name @trip}
meta name="description" content="..."
meta name="keywords" content="..."
body
div
...
- cache "trips/show/viewer/#{@trip.fragment_cache_key}", :expires_in => 1.day do
- @trip.eager_load_all
Tip: I added an eager_load_all method to trip to avoid n+1 problems when the cache does not hit:
def eager_load_all
ActiveRecord::Associations::Preloader.new([self], {:trip_days => [:weather_station_data, :nodes => [:entry, :notes => [:photo, :video, :audio]]]}).run
end
Tip 1: Conditional fragment caching
Unlike Caches_Action, Rails does not support conditional fragment caching. For example, if you want to give a fragment cache to an unused user, and the user does not use it, you can rewrite the helper to do just that:
def cache_if (condition, name = {}, cache_options = {}, &block) if condition cache(name, cache_options, &block) else yield end end- cache_if ! user_signed_in? , "xxx", :expires_in => 1.day do
Tip 2: Automatic updates of associated objects
The update_at timestamp is used as the cache key. You can add the touch option to the associated object to automatically update the associated object timestamp. For example, when updating or deleting comments on articles, you can automatically update the associated object timestamp.
class Article has_many :commentsendclass Comment belongs_to :article, :touch => trueend
5. Data query cache
Generally speaking, performance bottlenecks of Web applications occur in DB IO. Data query caching can greatly improve the overall response time by reducing the number of database queries.
There are two types of data query cache:
A. Cache within the same request period
As an example of displaying a list of articles, output the title and category of articles as follows
# controller
def index
@articles = Article.first(10)
end# view- @articles.each do |article|
h1 = article.name
span = article.category.name
Ten similar SQL queries will occur:
SELECT `categories`.* FROM `categories` WHERE `categories`.`id` = ?
Rails has query cache built in (https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb ), the same SQL query will be cached if there are no UPDATE /delete/ INSERT operations within the same request cycle. If the article category is the same, the database will be actually queried only once.
If article categories are different, N+1 query problems (a common performance bottleneck) can occur, Rails recommends using Eager Loading Associations ( http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations )
def index
@articles = Article.includes(:category).first(10)
end
The query statement becomes
SELECT `categories`.* FROM `categories` WHERE `categories`.`id` in (? ,? ,? ...).
B. Caching across request cycles
The performance optimization brought by the same request cycle cache is very limited. Most of the time, we need to use cross-request cycle cache, cache some commonly used data (such as User model). For active Record, use a unified query interface to fetch cache. This is easy to implement using callback to expire the cache, and there are some off-the-shelf gems to use.
For example identity_cache (https://github.com/Shopify/identity_cache)
class User < ActiveRecord::Base include IdentityCacheendclass Article < ActiveRecord::Base include IdentityCache Cached_belongs_to :userend# all hits cache user.fetch (1) article.find (2).user
The gem has the advantage of simple code implementation, flexible cache Settings, and easy to expand, but the disadvantage is that it requires a different fetch method name (fetch) and additional relational definitions.
If you want to be in countless according to cache the application of seamless join caching capabilities, recommend @ hooopo do second_level_cache (https://github.com/hooopo/second_level_cache).
Class User < ActiveRecord::Base acts_as_cached(:version => 1, :expires_in => 1.week)end# User.find(1)# no need to add a different belongs_to definition Article. Find (2).user
The implementation principle is to extend the Active Record underlying AREL SQL AST processing (https://github.com/hooopo/second_level_cache/blob/master/lib/second_level_cache/arel/wheres.rb)
It has the advantage of seamless access, but the disadvantage is that it is difficult to expand and cannot be cached for queries that only fetch a small number of fields.
6. Database cache
In the editor
The six caches are distributed at different locations on the client side to the server side, and the time savings are ranked from most to least.
From: Ruby China
Link: https://ruby-china.org/topics/19389
Baichuan.taobao.com is the wireless open platform of Alibaba Group. Through the opening of “technology, business and big data”, baichuan.taobao.com provides high cohesion, open, industry-leading technology product matrix, mature business components and perfect service system in mobile scenes. Help mobile developers quickly build apps, accelerate the process of APP commercialization, and empower mobile developers and mobile entrepreneurs in an all-round way.
About Alibaichuan 