preface
The CACHING mechanism of HTTP can be used to improve and optimize the efficiency of network requests. This will also be useful for project optimization, so this article will summarize the HTTP caching mechanism. I’ll also look at okHTTP source code to see how okHTTP implements this caching mechanism.
This article is based on OKHTTP 3.14.9
Github address: github.com/square/okht…
Gradle dependencies: Implementation Group: ‘com.squareup.okhttp3’, name: ‘okhttp’, version: ‘3.14.9’
For this article, see: Http caching strategy
Okhttp.Cache
Let’s start with how cache management is set up and used.
val okHttpClient = OkHttpClient.Builder()
.cache(Cache(this.cacheDir, 20 * 1024 * 1024))
.build()
// OkhttpClient.Builder.java
public Builder cache(@Nullable Cache cache) {
this.cache = cache;
this.internalCache = null;
return this;
}
Copy the codeWhen creating an OkHttpClient, you can set an okHttp3. Cache type. The parameters passed in are the path to the file stored in the Cache, and the maximum capacity.
// Cache.java
final InternalCache internalCache = new InternalCache() {
@Override public @Nullable Response get(Request request) throws IOException {
return Cache.this.get(request);
}
@Override public @Nullable CacheRequest put(Response response) throws IOException {
return Cache.this.put(response);
}
@Override public void remove(Request request) throws IOException {
Cache.this.remove(request);
}
@Override public void update(Response cached, Response network) {
Cache.this.update(cached, network);
}
@Override public void trackConditionalCacheHit(a) {
Cache.this.trackConditionalCacheHit();
}
@Override public void trackResponse(CacheStrategy cacheStrategy) {
Cache.this.trackResponse(cacheStrategy); }};Copy the codeThe internalCache attribute in a Cache is the interface provided to external calls to retrieve or save the Cache.
// OkhttpClient.java
@Nullable InternalCache internalCache(a) {
returncache ! =null ? cache.internalCache : internalCache;
}
Copy the codeOkhttpClient provides a method to get the internalCache attribute.
// RealCall.java
Response getResponseWithInterceptorChain(a) throws IOException {
// Build a full stack of interceptors.
List<Interceptor> interceptors = new ArrayList<>();
interceptors.addAll(client.interceptors());
interceptors.add(new RetryAndFollowUpInterceptor(client));
interceptors.add(new BridgeInterceptor(client.cookieJar()));
interceptors.add(new CacheInterceptor(client.internalCache()));
interceptors.add(new ConnectInterceptor(client));
Copy the codeWhen a network request, getResponseWithInterceptorChain () method will build Okhttp chain of responsibility, will be introduced to the then create CacheInterceptor internalCache object, caching policy for subsequent use.
So Okhttp caching mechanism processing, will occur on a CacheInterceptor. Intercept method.
// CacheInterceptor.java
final @Nullable InternalCache cache;
public CacheInterceptor(@Nullable InternalCache cache) {
this.cache = cache;
}
@Override public Response intercept(Chain chain) throws IOException { Response cacheCandidate = cache ! =null
? cache.get(chain.request())
: null;
long now = System.currentTimeMillis();
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
Copy the codeAs you can see from the code above, when the responsibility chain reaches the CacheInterceptor on a request, it first fetches a cached response from the cache. The cache.get method is the internalCache object’s get method.
A available cacheResponse is obtained by creating a CacheStrategy object and calling its GET method.
public CacheStrategy get(a) {
CacheStrategy candidate = getCandidate();
if(candidate.networkRequest ! =null && request.cacheControl().onlyIfCached()) {
// We're forbidden from using the network and the cache is insufficient.
return new CacheStrategy(null.null);
}
return candidate;
}
Copy the codeIn the cacheStrategy.get method, the getCandidate() method is called, so the getCandidate() method is the embodiment of the cache policy.
In this paper, the subsequent will combine CacheInterceptor. Intercept and getCandidate () method is introduced the cache mechanism.
Strong cache
HTTP caching mechanisms can be roughly divided into strong caching and comparative caching, starting with strong caching.
SequenceDiagram Client ->> Cache: Does the request have a compliant cache? Cache ->> Client: If there is an unexpired cache, return directly.Strong-cached fields include Expires and cache-control, where cache-control takes precedence over Expires.
Expires
This field is easy to understand and identifies the expiration time of the cache, which is the time after which the cache expires.
Cache-Control
You might see cache-control in the following format:
cache-control: public, max-age=7200
Copy the codeThis can be thought of as a custom type that contains multiple fields to form the effect of a configuration. Defined in Okhttp as the CacheControl class. Cache-control:
public
- Indicates that a response can be cached by any object (including the client sending the request, a proxy server such as a CDN, and so on), even content that is not normally cacheable (for example, the response does not have a Max-age directive or Expires header).
private
- Indicates that the response can only be cached by a single user and not as a shared cache (that is, the proxy server cannot cache it). A private cache can cache the response content.
no-cache
- You can cache locally, but each time a request is made, the server must verify that the local cache can be used only if the server allows it (that is, a negotiated cache is required).
no-store
- Do not cache the contents of client requests or server responses. Each time, you must request the server to retrieve the contents again
max-age
- Set the maximum period for which the cache is stored, after which the cache is considered expired (in seconds)
Okhttp source code
// CacheStrategy.Factory
public Factory(long nowMillis, Request request, Response cacheResponse) {
this.nowMillis = nowMillis;
this.request = request;
this.cacheResponse = cacheResponse;
if(cacheResponse ! =null) {
this.sentRequestMillis = cacheResponse.sentRequestAtMillis();
this.receivedResponseMillis = cacheResponse.receivedResponseAtMillis();
Headers headers = cacheResponse.headers();
for (int i = 0, size = headers.size(); i < size; i++) {
String fieldName = headers.name(i);
String value = headers.value(i);
if ("Date".equalsIgnoreCase(fieldName)) {
servedDate = HttpDate.parse(value);
servedDateString = value;
} else if ("Expires".equalsIgnoreCase(fieldName)) {
expires = HttpDate.parse(value);
} else if ("Last-Modified".equalsIgnoreCase(fieldName)) {
lastModified = HttpDate.parse(value);
lastModifiedString = value;
} else if ("ETag".equalsIgnoreCase(fieldName)) {
etag = value;
} else if ("Age".equalsIgnoreCase(fieldName)) {
ageSeconds = HttpHeaders.parseSeconds(value, -1); }}}}// getCandidate()
private CacheStrategy getCandidate(a) {... CacheControl responseCaching = cacheResponse.cacheControl();long ageMillis = cacheResponseAge();
long freshMillis = computeFreshnessLifetime();
if(requestCaching.maxAgeSeconds() ! = -1) {
freshMillis = Math.min(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds()));
}
long minFreshMillis = 0;
if(requestCaching.minFreshSeconds() ! = -1) {
minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds());
}
long maxStaleMillis = 0;
if(! responseCaching.mustRevalidate() && requestCaching.maxStaleSeconds() ! = -1) {
maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds());
}
if(! responseCaching.noCache() && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) { Response.Builder builder = cacheResponse.newBuilder();if (ageMillis + minFreshMillis >= freshMillis) {
builder.addHeader("Warning"."110 HttpURLConnection \"Response is stale\"");
}
long oneDayMillis = 24 * 60 * 60 * 1000L;
if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) {
builder.addHeader("Warning"."113 HttpURLConnection \"Heuristic expiration\"");
}
return new CacheStrategy(null, builder.build()); }...private long cacheResponseAge(a) {
longapparentReceivedAge = servedDate ! =null
? Math.max(0, receivedResponseMillis - servedDate.getTime())
: 0;
longreceivedAge = ageSeconds ! = -1
? Math.max(apparentReceivedAge, SECONDS.toMillis(ageSeconds))
: apparentReceivedAge;
long responseDuration = receivedResponseMillis - sentRequestMillis;
long residentDuration = nowMillis - receivedResponseMillis;
return receivedAge + responseDuration + residentDuration;
}
private long computeFreshnessLifetime(a) {
CacheControl responseCaching = cacheResponse.cacheControl();
if(responseCaching.maxAgeSeconds() ! = -1) {
return SECONDS.toMillis(responseCaching.maxAgeSeconds());
} else if(expires ! =null) {
longservedMillis = servedDate ! =null
? servedDate.getTime()
: receivedResponseMillis;
long delta = expires.getTime() - servedMillis;
return delta > 0 ? delta : 0;
} else if(lastModified ! =null
&& cacheResponse.request().url().query() == null) {
// As recommended by the HTTP RFC and implemented in Firefox, the
// max age of a document should be defaulted to 10% of the
// document's age at the time it was served. Default expiration
// dates aren't used for URIs containing a query.
longservedMillis = servedDate ! =null
? servedDate.getTime()
: sentRequestMillis;
long delta = servedMillis - lastModified.getTime();
return delta > 0 ? (delta / 10) : 0;
}
return 0;
}
Copy the codeThe above code roughly sums up:
CacheStrategytheFactoryMethods usinginCacheInterceptorThe incoming request for this launchrequestAnd the corresponding cachedresponse. There will bereadcacheResponsetheExpiresField and the date the response was last receivedDateIs used to calculate whether strong caching is valid. Of course, there are also some fields that are used to compare caches, which we’ll talk about later.getCandidate()Method will eventually return oneCacheStrategyObject, which can be understood as arequestRequest object and a processed cached responsecacheResponse.getCandidate()Method getscacheResponsethecachecontrolConfigure, parse andCalculates whether the cache is valid.- In getCandidate(), if the cache is finally judged to be validreturn
return new CacheStrategy(null, builder.build());Theta means that this is thetaStrong cachestrategyTo take effect, return directlyThe cached response. - It’s worth mentioning that In computeFreshnessLifetime(), Cachecontrol’s maxAge is judged first and expires is judged only when it doesn’t exist, which also indicates the priority between them.
/ / CacheInterceptor. Intercept () method
// If we don't need the network, we're done.
if (networkRequest == null) {
return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
}
Copy the codeEventually return to CacheInterceptor. The intercept (), if the judge return networkRequest is empty, is that can be returned directly in the cache cacheResponse, no longer a network request.
Compared to the cache
Contrast caching, also known as negotiated caching, is implemented by recording data changes in a field and asking the server whether the cache or updated data is needed.
SequenceDiagram Client ->> Server: Request server with cache flag ->> Client: Cache valid, return 304 Client ->> Cache: Cache cache ->> Client: cached responseThe comparison of the specific process of caching can be seen in the figure above. When the client requests the server, it carries a flag, and the server determines whether the cache is valid by judging the flag. If so, it will return a specific status code 304. Otherwise, return the latest data, HTTP status code 200.
Comparison caching is implemented in the following two ways:
- Last-Modified / If-Modified-Since
- Etag / If-None-Match
Last-Modified / If-Modified-Since
When the server is first requested, its response carries a last-Modified field that identifies when the data was Last Modified. In the next request, the client will carry the last-modified value of the response cached Last time to the IF-Modified-since field of the request and provide it to the server for judgment.
Etag / If-None-Match
This is easy to understand, replacing the Last-Modified mode with a tag. In my opinion, the benefit of this method is to effectively solve the problem of frequently modifying data or modifying the time of the card point. Because of the accuracy of time, it is possible to misjudge the validity of data by time, which is not accurate enough.
Ps: Etag/if-none-match has a higher priority than last-modified/if-modified-since.
Okhttp source code implementation
// getCandidate()
String conditionName;
String conditionValue;
if(etag ! =null) {
conditionName = "If-None-Match";
conditionValue = etag;
} else if(lastModified ! =null) {
conditionName = "If-Modified-Since";
conditionValue = lastModifiedString;
} else if(servedDate ! =null) {
conditionName = "If-Modified-Since";
conditionValue = servedDateString;
} else {
return new CacheStrategy(request, null); // No condition! Make a regular request.
}
Headers.Builder conditionalRequestHeaders = request.headers().newBuilder();
Internal.instance.addLenient(conditionalRequestHeaders, conditionName, conditionValue);
Request conditionalRequest = request.newBuilder()
.headers(conditionalRequestHeaders.build())
.build();
return new CacheStrategy(conditionalRequest, cacheResponse);
Copy the codeBack in the getCandidate() method, after experiencing the above strong cache failure, the request header assembly of the comparison cache is performed. Here you can see that eTag is checked first, and if not, lastModified is added and the last modification time is added. Etag/if-none-match has a higher priority than last-modified/if-modified-since.
// class CacheInterceptor
Response networkResponse = null;
try {
networkResponse = chain.proceed(networkRequest);
} finally {
// If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null&& cacheCandidate ! =null) { closeQuietly(cacheCandidate.body()); }}// If we have a cache response too, then we're doing a conditional get.
if(cacheResponse ! =null) {
if (networkResponse.code() == HTTP_NOT_MODIFIED) {
Response response = cacheResponse.newBuilder()
.headers(combine(cacheResponse.headers(), networkResponse.headers()))
.sentRequestAtMillis(networkResponse.sentRequestAtMillis())
.receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
networkResponse.body().close();
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
cache.trackConditionalCacheHit();
cache.update(cacheResponse, response);
return response;
} else{ closeQuietly(cacheResponse.body()); }}Copy the codeAfter returning to the CacheInterceptor, the cacheResponse data is used to respond to a network request with a status code of HTTP_NOT_MODIFIED(304).
Okhttp cache mechanism flow
Here is a diagram to illustrate the flow of caching:
- to
Strong cacheThe judgment, ifIf it is valid, return directly. - If the strong cache is invalid, the cacheResponse checks whether it carries an Etag field. If it does, the cacheResponse is added to if-none-match of the request for cache comparison.
- if
EtagfieldThere is no,judgecacheResponseWhether to carryLast-ModifiedFields, if any, are added to this timerequesttheIf-Modified-SinceDo comparison caching. - The process is described above
getCandidate()Methods are reflected.