How to implement an IOS network monitoring component

This article is published by netease Cloud community authorized by the author Zhu Zhiqiang.

Mobile Application Monitor IOS component design technology sharing

background

Application Performance Management (APM) is a popular Internet industry in recent years, and Mobile Application Monitor (MAM) is one of its core functions. APM monitors and optimizes enterprise critical applications to improve the reliability and quality of enterprise applications, ensure good services for users, and reduce the TOTAL cost of IT ownership (TCO). An enterprise’s business-critical applications are high-performance, competitive and commercially successful, so enhanced application performance management can yield significant business benefits. Mature products include:

The target

• Network statistics component of aN iOS client that collects HTTP request data for iOS apps, such as request time, data, and errors

• Design a reusable framework to facilitate the subsequent addition of monitoring content such as frame rate and user experience

• The impact on the application is as small as possible, easy to use

The design model

Data processing is divided into 4 steps:

Data collection, data assembly, data persistence, data delivery

Thread model:

The data collection is responsible for initializing the MAMDataBuilder, completing the data assembly and database insert operations in the persistence layer queue.

When the conditions for sending data are met, the persistence layer queue first searches for data from the database, and then sends data in the sending layer queue. After sending, the data is deleted in the persistence layer queue, and then the next data is processed.

The following figure illustrates the program execution using a graph, with the gray rectangle representing the API interface

This paper mainly for common network technology interception technology to do a comprehensive detailed explanation and analysis.

Data collection Hooker

Collect data for HTTP requests of NSURLConnection and CFNetwork, the main network technologies of IOS

The hook NSURLConnection

Interception of messages sent to Objective-C objects

• Technical background

• Runtime Objective-C is a Runtime language that, as much as possible, defers the decision of code execution from compile and link time to Runtime. This gives you a lot of flexibility in writing code, such as being able to forward messages to the object you want, or swap implementations of a method at will. Method Swizzling is used to achieve the purpose of hook by the way of exchange Method implementation.

• When dynamic binding is compiled, we don’t know which code will be executed, and we can only determine which code to execute when we execute it, by looking it up with selector. The method type in Objective-C is SEL. Instance object performSelector, in the respective message selector/implementation address method list to find the specific method implementation (IMP), and then use this method implementation to execute the specific implementation code.

• IMP type IMP is the function pointer to the execution code that the message eventually calls. It can be interpreted as every objective-C method is converted to C function at compile time. IMP is the function pointer to this C function. An Objective-C method:

  -(void)setFilled:(BOOL)arg;Copy the code

  Its Objective-C call would look like this:

  [aObject setFilled:YES];Copy the code

  Call the method of base class NSObject – (IMP)methodForSelector:(SEL)aSelector to get IMP

  void (*setter)(id, SEL, BOOL);  
  setter = (void (*)(id, SEL, BOOL))[self methodForSelector:@selector(setFilled:)];Copy the code

  The equivalent C call is to IMP (function pointer) :

  setter(self, @selector(setFilled:), YES)Copy the code

• Method Swizzling
  
  Normally, we have no way of knowing when a system Method is called, but by replacing the code implementation of the system Method, we can get the timing of the system Method. This is Method Swizzling!
  
  In the following figure, modify the corresponding IMP selector to save the original IMP function, which implements the system call hook.

• Swizzling Method Swizzling

  BOOL HTSwizzleMethodAndStore(Class class, BOOL isClassMethod, SEL original, IMP replacement, IMP* store) {
    IMP imp = NULL;
    Method method ;  if (isClassMethod) {
        method= class_getClassMethod(class, original);
    }else{
        method= class_getInstanceMethod(class, original);
    }  if (method) {
        imp = method_setImplementation(method,(IMP)replacement);      if (!imp) {
            imp = method_getImplementation(method);
        }
    }else{
        MAMLog(@"%@:not found%@!!!!!!!!",NSStringFromClass(class),NSStringFromSelector(original));
    }  if(imp && store) { *store = imp; }// Place the original method in storereturn(imp ! = NULL); }Copy the code

  Declare function pointer IMP store, implement function MAM IMP

  static NSURLConnection * (*Original_connectionWithRequest)(id self, SEL _cmd, NSURLRequest *request, id delegate); static NSURLConnection * MAM_connectionWithRequest(id self, SEL _cmd, NSURLRequest *request, Id result = Original_connectionWithRequest(self, _cmd, request, hookDelegate); // Get the timing of the system method call herereturn result;
  }Copy the code

  Swizzling is called after the program starts

  HTSwizzleMethodAndStore(NSClassFromString(@"NSURLConnection"),
                            YES,                          @selector(connectionWithRequest:delegate:),
                            (IMP)MAM_connectionWithRequest,
                            (IMP *)&Original_connectionWithRequest);Copy the code

• Monitoring of the delegate model
  
  The Runtime replacement method requires a class name, but the delegate class of NSURLConnection is not specified. If you still use Method Swizzling to intercept delegate messages, each additional class that uses NSURLConnectionDelegate would need to dynamically declare IMP Store and MAM IMP, which would be inefficient.
  
  The solution is to replace the delegate of NSURLConnection with a proxy delegate. Just make sure the proxy forwards any network callbacks it receives to the original delegate.

The hook CFNetwork

Interception of C function calls

• Technical background

• Use the Dynamic Loader Hook library function —- fishhook
  
  Dynamic Loader (DYLD) binds symbols by updating Pointers saved in the Mach-O file. Using it, you can modify the function pointer of a C function call at run time!
  
  fishhookThe procedure for finding a function symbol name is shown below
  
  
  
  In the figure above, 1061 is the offset for the Indirect Symbol Table. The stored Symbol Table has an offset of 16343.
  
  Fishhook changes the offset of the indirect Symbol Table. This changes the offset of the Symbol Table to indicate the Actual Symbol Name in the Symbol Table. Modify the pointer to the real calling function by modifying the real symbol name to achieve the purpose of hook.

• The Stream’s read size and Toll-Free Bridge CFNetwork use CFReadStreamRef for data passing and receive the server’s response using a callback function. The server data is retrieved by a callback function that, when notified of the presence of data in the stream, reads the data from the stream and stores it in client memory. Stream reading is not suitable for modifying the string table. This requires the hook system to use the read function, and the system read function is not only called by the network request stream, but also all the file processing, and hook a frequently called function is not desirable. For a stream of type C, it is really difficult. But for an object, there are many ways to use it. Yes, use the Toll-free Bridge! With this, you can convert the CFReadStreamRef type directly to an NSInputStream object!! The Toll-Free Bridge converts Cocoa objects to CoreFoundation.

  CFIndex CFReadStreamRead(CFReadStreamRef readStream, UInt8 *buffer, CFIndex bufferLength) {
  CF_OBJC_FUNCDISPATCH2(__kCFReadStreamTypeID, CFIndex, readStream, "read:maxLength:", buffer, bufferLength);Copy the code

  The first line of the function calls the Cocoa methodread:maxLength:, which confirms the toll-Free Bridge’s implementation mechanism — a library in Objective-C that can be called in pure C.
  
  Finally, design the stream to be monitored like this:
  
  This has successfully turned the problem of hook a C function into a problem of hook an Objective-C method. However, NSInputStream is still a low-level public class that needs to hook the system’s read method. Can we hook only a stream object?
  
  Yes, Trampoline!

• When an instance object receives a message, but no implementation of the message is found, the following two methods are called, giving the developer the option to forward the message

  -(NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector;
  -(void)forwardInvocation:(NSInvocation *)anInvocation;Copy the code

  By borrowing the forwarding mechanism, you can hook a given object: design a Proxy class that inherits from NSObject and holds an NSInputStream, called OriginalStream. Use the above methods to forward messages to the Proxy to OriginalStream. This way, all messages sent to the Proxy are handled by OriginalStream. To get the stream size, rewrite the NSInputStream read method. This design of modifying the execution direction of a program is called a Trampoline, meaning a Trampoline, symbolizing bouncing the method back to the actual recipient. MAMNSStreamProxy core code:

  -(instancetype)initWithClient:(id*)stream
  {if(self = ! [super init]) { _stream = ! [stream retain]; }return self;
  }
  -(NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
  {return! [_stream methodSignatureForSelector:aSelector]; } -(void)forwardInvocation:(NSInvocation *)anInvocation { ! [anInvocation invokeWithTarget:_stream]; } -(NSInteger)read:(uint8_t *)buffer maxLength:(NSUInteger)len
  {
  NSInteger rv = [_stream read:buffer maxLength:len]; // Sizereturn rv; }Copy the code

• Method Swizzling: Method Swizzling: Method Swizzling: Method Swizzling

  static CFReadStreamRef(*original_CFReadStreamCreateForHTTPRequest)(CFAllocatorRef alloc, CFHTTPMessageRef request); /** * MAMNSInputStreamProxy holds original CFReadStreamRef, forwards messages to original CFReadStreamRef, in methodreadTo get the data size. * use original CFReadStreamRef as key, Save CFHTTPMessageRef request * / static CFReadStreamRefMAM_CFReadStreamCreateForHTTPRequest (CFAllocatorRef alloc, CFHTTPMessageRef request){CFReadStreamRef originalCFStream = original_CFReadStreamCreateForHTTPRequest(alloc, request); // Convert CFReadStreamRef to NSInputStream, save it in MAMNSInputStreamProxy, CFReadStreamRef NSInputStream* stream = (__bridge NSInputStream*)originalCFStream; MAMNSInputStreamProxy *outReadStream = ! [![MAMNSInputStreamProxy alloc] initWithStream:stream]; */ CFRelease(originalCFStream); */ CFRelease(originalCFStream); */ CFReadStreamRef result = (__bridge_retained CFReadStreamRef)((id)outReadStream);return result;
  }Copy the code

  Replace the function address with Fishhook

  save_original_symbols(); int bFishHookWork = rebind_symbols((struct rebinding! [1]) {{"CFReadStreamCreateForHTTPRequest", MAM_CFReadStreamCreateForHTTPRequest},},1);Copy the code

  void save_original_symbols(){
   original_CFReadStreamCreateForHTTPRequest = dlsym(RTLD_DEFAULT, "CFReadStreamCreateForHTTPRequest");
  }Copy the code

• Data interception model
  
  According to the calling method of CFNetwork API, the design model of fishhook and proxyStream to obtain C function is as follows:
  
  

For more information about netease’s technologies, products and operational experience, please visit netease Cloud Community.

Git uses asymmetric encryption and certificates.