preface

Recently in the company to do network related optimization, reorganize the previous cognition of HttpDNS and write this article, to build the HttpDNS scheme as a base, explain the actual mobile access code, because everyone’s implementation scheme is different, here is just throw diversion, not necessarily suitable for all projects.

introduce

When we initiate a request with a domain name, the DNS needs to be resolved into an IP address before initiating the request, so the stability of DNS domain name resolution is very important, is the first step of network request.

Domain name Resolution Process

By default, domain names are first queried by carriers’ LocalDNS. For example, a telecom user queries the LocalDNS of a telecom user and a mobile user queries the LocalDNS of a mobile user. Usually, the RESOLVED IP addresses of the two are different.

If the LocalDNS server is not matched, the DNS server is forwarded to the authoritative DNS server. For example, to access www.163.com, search the DNS root server to obtain the IP address of the.com domain server, and then search the.com server to obtain the IP address of the 163.com domain server. Finally, the 163.com domain server was used to obtain the correct IP address, and the IP address was cached in the LocalDNS server.

The overall process is shown as follows:

For more information about DNS, see The network logs of Nguyen Yifeng about DNS Principles

Problems with LocalNDS

With the increasing number of App users in different regions and operators, access failure or slow access often occurs. The following problems have been found after positioning.

LocalDNS fault

The carrier server is faulty and cannot send recursive query to the authoritative server. As a result, the resolution fails.

DNS hijacking

A third party hijacks the DNS server and tampers the resolution result. As a result, the client accesses an incorrect IP address to steal data or access data maliciously.

DNS resolution is returned by cache

LocalDNS caches the previous resolution results and does not access the authoritative DNS server when receiving the next resolution request. This ensures that users’ access traffic is absorbed on the local network or advertisements are inserted. If the IP address or port of the authoritative server is changed, the LocalDNS is not updated, causing an access failure.

Parsing forwarding by small operators

To save resources, small carriers directly send requests to other carriers for recursive resolution instead of sending requests to the authoritative DNS server. As a result, cross-network access slows down user access.

What is a HttpDNS

Since LocalDNS is problematic and uncontrollable, can it be bypassed and resolved by itself? The answer is yes. By maintaining a set of mapping between domain names and IP addresses on its own server, it does not perform DNS resolution through port 53 of LocalDNS, but directly initiates HTTP requests to port 80 of its own server to obtain IP, and then makes network requests directly through IP. This approach is HttpDNS.

Steps to initiate a business request:

  1. The client directly accesses the HttpDNS interface to obtain the optimal IP address matching the requested domain name.
  2. The client sends a service protocol request directly to the obtained IP address. Using Http requests as an example, you can send a standard Http request to the IP returned by HttpDNS by specifying the host field in the header.
  3. In consideration of fault tolerance, the LocalDNS request mode is reserved as an alternate solution.

HttpDNS advantages

  • DNS hijacking is fundamentally avoided because no resolution is initiated to LocalDNS.
  • Directly access through IP addresses, avoiding domain name resolution and improving user access speed.
  • It can sort the success rate of IP requests through algorithms on its own server, and screen out high-quality IP addresses, which increases the success rate of requests.

Network request implementation on iOS

HttpDNS overall solution needs the server and mobile terminal cooperate with each other, in the mobile terminal is mainly on the network request encapsulation, replace the domain name request, do no perception of the user, do a good job of caching and fault tolerance processing, and the successful/failed request log upload to the server; The server needs to maintain the mapping table between domain names and IP addresses, provide a delivery interface, and optimize the order based on client logs.

Next we explore some implementation steps.

Configure the server to deliver IP addresses

Request the configuration table from the server when the App starts or at a suitable time. The request here can replace the domain name with a fixed IP to avoid the domain name resolution process. The point to note here is that if you use IP requests, you need to specify the host field in the header.

NSString *host = "a.test.com";
[request setValue:host forHTTPHeaderField:@"Host"];
Copy the code

The configuration table to be delivered depends on actual requirements. For example:

{
    "service" : "Shenzhen Mobile"."enable" : 1."domainlist": [{"domain": "a.test.com"."ips" :  [
                        "222.66.22.111"."222.66.22.102"] {},"domain": "b.test.com"."ips" :  [
                        "202.29.13.214"]}}Copy the code

Encapsulating network requests

AFNetworking, the networking framework used here, is the basis for our encapsulation.

Request return to block / * * * / typedef void (^ YENetworkManagerResponseCallBack) (NSDictionary * response, NSDictionary * error); @interface YENetworkManager : NSObject + (nonnull instancetype)shareInstance; */ - (void)requestRemoteDNSList; /** * network request * @param URL request address * @param paraDic request input parameter {key: Value} * @ param method request type GET | POST * @ param timeoutInterval request timeout * @ param headersDic request header {key: Value} * @param callBack request result callBack */ - (void)requestWithUrl:(NSString *)url body:(NSDictionary *)paraDic method:(NSString) *)method timeOut:(NSTimeInterval)timeoutInterval headers:(NSDictionary *)headersDic callBack:(YENetworkManagerResponseCallBack)callBack; @endCopy the code

Two interfaces are exposed to pull DNS configurations and network requests, respectively. Part of the difference in network requests is that IP addresses are used to replace domain names before formal requests are initiated.

Pull configuration here directly from the local read, the actual project or should go to the background interface to request data.

- (void)requestRemoteDNSList {NSError*error = nil; NSData *data = [[NSData alloc] initWithContentsOfFile:[[NSBundle mainBundle] pathForResource:@"dns.json" ofType:nil]]; NSDictionary *dic = [NSJSONSerialization JSONObjectWithData:data options:NSJSONReadingAllowFragments error:&error]; NSArray *domainlist = dic[@"domainlist"]; NSMutableArray *tempDNSEntityArray = [[NSMutableArray alloc] initWithCapacity:0]; For (NSDictionary *domainDict in DomainList) {// Create entity and save YEDNSEntity *cdnsEntity = [YEDNSEntity yy_modelWithDictionary:domainDict]; [tempDNSEntityArray addObject:cdnsEntity]; } self.dnsEntityListCache = tempDNSEntityArray; // TODO: Whether to store in local database according to actual requirements}Copy the code

To convert an IP address, use the domain name as the key and search for the corresponding address in the cache. If a match is found, create a new request and complete:

  1. Replace the interface domain name with IP
  2. Add the domain name to the header host field
  3. Sets the Cookie of the original request to the new request
/ / determine whether support - (BOOL) supportHTTPDNS request: (NSURLRequest *) {/ / no DNS data does not handle the if (self) dnsEntityListCache) count = = 0) {return NO; } if ([request.url.scheme rangeOfString:@" HTTP "]. Location == NSNotFound) {return NO; } / / IP does not handle the if ([self isIPAddressString: request. The URL. The host]) {return NO; } return YES; } - (NSURLRequest *)transfromHTTPDNSRequest:(NSURLRequest *)request {if ([self supportHTTPDNS:request]) { YEDNSEntity *entity = [self queryDNSEntityWithDomain:request.URL.host]; if (entity == nil) { return request; } // create IP request NSMutableURLRequest *newURLRequest = request.mutablecopy; NSString *ipAddress = nil; If (entity.ips && entity.ips.count > 0 && (ipAddress = entity.ips.firstobject)) {// replace originalHost with IP NSString *originalHost =  request.URL.host; NSString *newUrlString = [newURLRequest.URL.absoluteString stringByReplacingFirstOccurrencesOfString:originalHost withString:ipAddress]; newURLRequest.URL = [NSURL URLWithString:newUrlString]; NSString *realHost = originalHost; [newURLRequest setValue:realHost forHTTPHeaderField:@"host"]; / / add the original domain name corresponds to the Cookie nsstrings * Cookie = [self getCookieHeaderForRequestURL: request the URL]; if (cookie) { [newURLRequest setValue:cookie forHTTPHeaderField:@"Cookie"]; } } return newURLRequest; } return request; }Copy the code

Then we get the request body of the new IP and send the request through AFNetworking.

- (void)requestWithUrl:(NSString *)url body:(NSDictionary *)paraDic method:(NSString *)method timeOut:(NSTimeInterval)timeoutInterval headers:(NSDictionary *)headersDic CallBack: (YENetworkManagerResponseCallBack) callBack {exception handling / / / / parameters... AFHTTPRequestSerializer *requestSerializer = [AFJSONRequestSerializer]; / / set the timeout requestSerializer. TimeoutInterval = timeoutInterval < 0? 10 :timeoutInterval; For (NSString *headerName in headersDic. AllKeys) {NSString *headerValue = [headersDic objectForKey:headerName]; [requestSerializer setValue:headerValue forHTTPHeaderField:headerName]; } // create originalRequest NSURLRequest *originalRequest = [requestSerializer requestWithMethod:method URLString:url parameters:[paraDic count] == 0 ? nil : paraDic error:nil]; / / HTTPDNS processing NSURLRequest * ipRequest = [self transfromHTTPDNSRequest: originalRequest]; // SessionManager [[YESessionTool shareInstance] getSessionManagerWithRequest:ipRequest callBack:^(YESessionManager * _Nonnull sessionManager) { [sessionManager dataTaskWithRequest:ipRequest uploadProgress:^(NSProgress * _Nonnull UploadProgress :^(NSProgress * _Nonnull downloadProgress) {// Not processed} completionHandler:^(NSURLResponse * _Nonnull response, id _Nullable responseObject, NSError * _Nullable error) { if (callBack) { if (error) { NSDictionary *errorDic = [NSDictionary dictionaryWithObject:error.description forKey:@"message"]; callBack(@{}, errorDic); } else {// NSDictionary *responseDict = [responseObject object jsondata]; if (responseDict! = nil && [responseDict isKindOfClass:[NSDictionary class]]) { callBack(responseDict, @{}); } else {NSDictionary *errorDic = [NSDictionary dictionaryWithObject:@" forKey:@"message"];  callBack(@{}, errorDic); } } } }]; }]; }Copy the code

Fault tolerant processing & burying point

When there is a problem with the IP request, we need to degrade the process and try the request again using the alternate IP address or domain name. In addition, it is better to upload the success or failure log after the request, so as to facilitate the server to analyze the availability of IP. We modify the above request response part:

// SessionManager [[YESessionTool shareInstance] getSessionManagerWithRequest:ipRequest callBack:^(YESessionManager * _Nonnull sessionManager) { [sessionManager dataTaskWithRequest:ipRequest uploadProgress:^(NSProgress * _Nonnull UploadProgress :^(NSProgress * _Nonnull downloadProgress) {// Not processed} completionHandler:^(NSURLResponse * _Nonnull response, id _Nullable responseObject, NSError * _Nullable error) { if (callBack) { if (error) { //TODO: Failure point upload / / downgrade request if ([self canDegradeForRequest: ipRequest. URL error: error]) {/ / remove the IP [the self removeIpInCacheWithDomain:originalRequest.URL.host ip:ipRequest.URL.host]; // Initiate a new request. [self requestWithUrl: URL Body :paraDic Method :method timeOut:timeoutInterval headers:headersDic callBack:callBack];  } else { NSDictionary *errorDic = [NSDictionary dictionaryWithObject:error.description forKey:@"message"];  callBack(@{}, errorDic); } } else { //TODO: Success buried point upload / / save cookies if (! [self isIPAddressString: originalRequest. URL. The host] &&! [originalRequest.URL.host isEqualToString:ipRequest.URL.host]) { NSDictionary *responseHeaderDict = ((NSHTTPURLResponse *)response).allHeaderFields; [self storageHeaderFields:responseHeaderDict forURL:ipRequest.URL]; } NSDictionary *responseDict = [responseObject objectFromJSONData]; if (responseDict! = nil && [responseDict isKindOfClass:[NSDictionary class]]) { callBack(responseDict, @{}); } else {NSDictionary *errorDic = [NSDictionary dictionaryWithObject:@" forKey:@"message"];  callBack(@{}, errorDic); } } } }]; }];Copy the code

Resolve the security certificate verification problem

Certificate verification includes IP request and domain name request. For common domain name request, you only need to set the SessionManager security policy.

- (void)setDomainNetPolicy: (YESessionManager *)manager request:(NSURLRequest *)request { AFSecurityPolicy *securityPolicy = [AFSecurityPolicy policyWithPinningMode:AFSSLPinningModeCertificate]; securityPolicy.validatesDomainName = YES; securityPolicy.allowInvalidCertificates = YES; NSString * cerPath = [[NSBundle mainBundle] pathForResource:CerFile ofType:@"cer"]; NSData * cerData = [NSData dataWithContentsOfFile:cerPath]; securityPolicy.pinnedCertificates = [NSSet setWithObject:cerData]; manager.securityPolicy = securityPolicy; }Copy the code

The IP request part is slightly more complicated. When we receive the server security authentication request, we use the real domain name and local certificate to verify. Provides setSessionDidReceiveAuthenticationChallengeBlock and setTaskDidReceiveAuthenticationChallengeBlock AFNetworking Method allows us to set the callback for authentication requests.

- (void)setIPNetPolicy: (YESessionManager *)manager request:(NSURLRequest *)request {// check whether domain name NSString *realDomain = exists [request.allHTTPHeaderFields objectForKey:@"host"]; If (realDomain = = nil | | realDomain. Length = = 0) {/ / without domain name does not verify the return; } // Verify the server credential [manager] through the client setSessionDidReceiveAuthenticationChallengeBlock:^NSURLSessionAuthChallengeDisposition(NSURLSession * _Nonnull session, NSURLAuthenticationChallenge * _Nonnull challenge, NSURLCredential *__autoreleasing _Nullable * _Nullable credential) { return [self handleReceiveAuthenticationChallenge:challenge credential:credential host:realDomain]; }]; [manager setTaskDidReceiveAuthenticationChallengeBlock:^NSURLSessionAuthChallengeDisposition(NSURLSession * _Nonnull session, NSURLSessionTask * _Nonnull task, NSURLAuthenticationChallenge * _Nonnull challenge, NSURLCredential *__autoreleasing _Nullable * _Nullable credential) { return [self handleReceiveAuthenticationChallenge:challenge credential:credential host:realDomain]; }]; } // Handle the authentication request callback - (NSURLSessionAuthChallengeDisposition)handleReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge*)challenge credential:(NSURLCredential**)credential host:(NSString*)host { NSURLSessionAuthChallengeDisposition disposition = NSURLSessionAuthChallengePerformDefaultHandling; If ([challenge. ProtectionSpace authenticationMethod isEqualToString: NSURLAuthenticationMethodServerTrust]) {/ / verify whether the domain name is trust  if ([self evaluateServerTrust:challenge.protectionSpace.serverTrust forDomain:host]) { *credential = [NSURLCredential credentialForTrust:challenge.protectionSpace.serverTrust]; if (*credential) { disposition = NSURLSessionAuthChallengeUseCredential; } else { disposition = NSURLSessionAuthChallengePerformDefaultHandling; } } else { disposition = NSURLSessionAuthChallengeCancelAuthenticationChallenge; } } else { disposition = NSURLSessionAuthChallengePerformDefaultHandling; } return disposition; EvaluateServerTrust :(SecTrustRef)serverTrust forDomain:(NSString *)domain {AFSecurityPolicy *securityPolicy = [AFSecurityPolicy policyWithPinningMode:AFSSLPinningModeCertificate]; securityPolicy.validatesDomainName = YES; securityPolicy.allowInvalidCertificates = YES; NSString * cerPath = [[NSBundle mainBundle] pathForResource:CerFile ofType:@"cer"]; NSData * cerData = [NSData dataWithContentsOfFile:cerPath]; securityPolicy.pinnedCertificates = [NSSet setWithObject:cerData]; return [securityPolicy evaluateServerTrust:serverTrust forDomain:domain]; }Copy the code

The flow chart of the total

conclusion

This article provides a brief introduction to HttpDNS and domain name resolution issues. The code has been placed in ios developtools-network for reference only.

At present, the implementation is not a perfect solution to the network request coupling. Later, there is time to supplement the decoupling of HTTPDNS module and WKWebview and AVplayer processing, please wait for 😂.

About Me

  • The Denver nuggets

  • Github

Refer to the link

  • A new approach to global precise traffic scheduling -HttpDNS service details
  • Probably the most complete iOS HttpDns integration solution
  • DNS high availability for App domain name hijacking
  • Comprehensive understanding of DNS and HTTPDNS
  • Resolve the certificate verification problem of HTTPDNS + HTTPS
  • What is CDN? What does it have to do with DNS? Hetoto’s blog
  • DNS optimization practice for APP network optimization