directory

  1. Introduction to Basic Knowledge
    • Linux EventFD event wait/response mechanism
    • Linux IO multiplexing epoll
  2. Android message mechanism Native layer analysis
    • NativeInit process
    • NativePollOnce process
    • NativeWake process
  3. data
  4. harvest

One thing that was missing from the previous article was the collection of ThreadLocal, which can cause memory leaks. The key in threadLocalmap. Entry has a weak reference to ThreadLocal, while the value needs to allocate memory on the heap. When the strong reference of the key is broken, it will be recycled during GC, so the key is null, but the value still exists. If not handled properly, memory leaks can occur.

Image from: Thoroughly clarifying ThreadLocal vs. weak references

If the key is null, when will the value be cleared? The designers of ThreadLocal take this issue into account by clearing a value with a null key each time a ThreadLocal’s set/get method is called. But who knows when set/get will be called next, which is a passive cleanup. ThreadLocal also provides a remove method that can be automatically removed. Remember to call remove when ThreadLocal is no longer in use. The sample is as follows

public class ExampleUnitTest { private static ThreadLocal<String> sStrThreadlocal = new ThreadLocal<>(); private static ThreadLocal<Integer> sIntegerThreadLocal = new ThreadLocal<>(); @test public void testThreadLocal(){sstrThreadlocal.set ("aaa"); String value = sStrThreadlocal.get(); sIntegerThreadLocal.set(1); Integer intValue = sIntegerThreadLocal.get(); System.out.println("111 curThreadId="+Thread.currentThread()+" strthreadLocalValue="+value +" intThreadLocalValue="+intValue); try { Thread.sleep(100); } catch (InterruptedException e) { e.printStackTrace(); } // The main thread value value = sstrThreadlocal.get (); intValue = sIntegerThreadLocal.get(); System.out.println("444 curThreadId="+Thread.currentThread()+" strthreadLocalValue="+value +" intThreadLocalValue="+intValue); Threadlocalmap. Entry's key is a weak reference to ThreadLocal, but the memory space occupied by the value is still there. // There are two scenarios in which the map is reclaimed. When the set or GET method is called again, the map will insist whether there is an Entry with an empty key (because the key is a weak reference, and after the external strong reference dependency is broken, the key will be reclaimed during gc, and then null). If there is an Entry with an empty key, the map will be cleared. ThreadLocal does not know when to call set or get again, so it guarantees that memory leaks will occur if it does not call set or get again. Sstrthreadlocal.remove (); sstrThreadlocal.remove (); sIntegerThreadLocal.remove(); value = sStrThreadlocal.get(); intValue = sIntegerThreadLocal.get(); System.out.println("after remove 444 curThreadId="+Thread.currentThread()+" strthreadLocalValue="+value +" intThreadLocalValue="+intValue); }}Copy the code

The corresponding output is

111 curThreadId=Thread[main,5,main] strthreadLocalValue=aaa intThreadLocalValue=1
444 curThreadId=Thread[main,5,main] strthreadLocalValue=aaa intThreadLocalValue=1
after remove 444 curThreadId=Thread[main,5,main] strthreadLocalValue=null intThreadLocalValue=null
Copy the code

Let’s start with the analysis

We have learned from the previous two analyses of Android message mechanism what Native interactions are involved in MessageQueue, such as MessageQueue initialized by nativeInit of Native layer. NativePollOnce block; NativeWake sensei.

The question is, why? Doesn’t the direct Java layer manage and maintain message queues and read and distribute messages? What does nativePollOnce really mean? The question keeps bothering me. Through repeatedly check the source code, finally understand its process. Before implementing the message mechanism Native layer, let’s take a look at some Linux basics. Otherwise, it’s easy to get stuck.

I. Introduction to basic knowledge

1.1 Linux EventFD Wait/response mechanism

Common interprocess/thread communication mechanisms in Linux include pipes, semaphores, message queues, signals, shared memory, sockets, etc. Pipes and sockets are mainly used for interprocess/thread notification/waiting. Since Linux 2.6.27, eventFD has been added for communication between processes or threads. Eventfd is used for communication between processes and threads, which is more efficient than PIPE.

1.2 Linux I/O Multiplexing epoll

Epoll is the most efficient IO multiplexing mechanism in Linux. It can monitor multiple descriptors at the same time, and when a descriptor is ready (read or write), it immediately informs the program to read or write

Epoll has three methods epoll_create(), epoll_ctl(), and epoll_wait()_

** epoll_CREATE: Creates an epoll handle **

Int epoll_create (int size); Size refers to the number of listening descriptors. Now the kernel supports dynamic expansion. This value only means the number of FDS allocated for the first time. After the epoll handle is created, a fd value is occupied. After using epoll, you must call close() to turn it off, or you may run out of FDS.Copy the code

** epoll_ctl: Performs the operation on the file descriptor (fd) to be listened on **

Int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event); For example, EPOLL_CTL_ADD fd: the file descriptor to be listened on. Epoll_event: the event to be monitoredCopy the code

** epoll_wait: wait for an event **

int epoll_wait(int epfd, struct epoll_event * events, int maxevents, int timeout); Waiting for an event to arrive, there are three conditions that trigger a return: 1. An error occurs and the return value is negative 2. 3. Events are detected on some file handlesCopy the code

The thread triggers epoll_wait(), which is blocked, and another thread wakes up by writing data to the eventfd, making the descriptor readable, and epoll returns. _

Ii. Simple analysis of Android message mechanism Native layer

Let’s start by reviewing the Java layer MessageQueue’s calls to add and fetch messages from queues

//android.os.MessageQueue#enqueueMessage boolean enqueueMessage(Message msg, long when) { ...... synchronized (this) { msg.markInUse(); msg.when = when; Message p = mMessages; boolean needWake; / / if the news list is empty, or inserted into the Message than the news list first to perform more early, if directly inserted into the head (p = = null | | the when = = 0 | | the when < p.w hen) {MSG. Next = p; mMessages = msg; needWake = mBlocked; } else {// Otherwise find the right place in the list to insert // Normally there is no need to wake up the event queue, unless the head of the list is a synchronization barrier, And this message is the first asynchronous message needWake = mBlocked && p.target == null && msg.isasynchronous (); // The list introduces a prev variable that points to p as well as message (if it is executed inside the for loop for the first time), then moves P to next and compares it with the message to insert when message prev; for (;;) { prev = p; p = p.next; if (p == null || when < p.when) { break; } if (needWake && p.isAsynchronous()) { needWake = false; } } msg.next = p; prev.next = msg; } // If an asynchronous message is inserted and the first message in the message list is a synchronous barrier message. // Or the list of messages has only one Message that has just been inserted, and mBlocked is true, i.e., it is being awakened after receiving a Message. Messagequeue.next blocks nativePollOnce if (needWake) {nativeWake(mPtr); } } return true; } / / android. OS. MessageQueue# next Message next () {/ / pointer to native NativeMessageQueue layer of final long PTR = mPtr; if (ptr == 0) { return null; }... for (;;) {// Block operations when waiting for nextPollTimeoutMillis, or when the message queue is awakened //nativePollOnce is used to "wait" until the next message is available. If it takes a long time during this call, it indicates that the corresponding thread has no actual work to do, or that the Native layer message has time-consuming operations performing nativePollOnce(PTR, nextPollTimeoutMillis). }Copy the code

It can be seen that in MessageQueue#next, native method nativePollOnce, which is blocked by the course, is called. In MessageQueue#enqueueMessage, native method nativeWake is called if it needs to be woken up. How does the problem block, how does it wake up, why is it designed this way, can’t you just do it in the Java layer?

With such confusion, I began to analyze and learn the message mechanism of Native layer.

2.1 MessageQueue Init Process

Android Message mechanism 2-Handler(Native layer)

Call Native method initialization, return value is Native layer NativeMessageQueue pointer address

//android.os.MessageQueue#MessageQueue MessageQueue(boolean quitAllowed) { mQuitAllowed = quitAllowed; MPtr = nativeInit(); }Copy the code

android_os_MessageQueue_nativeInit

// Java native method nativeInit jNI method, return type long, Static jlong android_os_MessageQueue_nativeInit(JNIEnv* env, Jclass clazz) {//NativeMessageQueue is an inner class NativeMessageQueue* NativeMessageQueue = new NativeMessageQueue(); // Return the NativeMessageQueue pointer address...... return reinterpret_cast<jlong>(nativeMessageQueue); }Copy the code

NativeMessageQueue constructor

Looper creation of Native layer will be performed. The Java layer creates a Looper and then creates a MessageQueue, whereas the Native layer does the opposite, creating a NativeMessageQueue and then creating a Looper.

// core/jni/android_os_MessageQueue. CPP //NativeMessageQueue constructor //Java layer create Looper and then create MessageQueue, //Native layer just the opposite, Create NativeMessageQueue first and then create Looper //MessageQueue is in the Java layer and Native layer has a tight connection, / / but Native layer which has nothing to do with Java layer which NativeMessageQueue: : NativeMessageQueue () : MPollEnv (NULL), mPollObj(NULL), mExceptionObj(NULL) { MyLooper () mLooper = Looper::getForThread(); If (mLooper == NULL) {// If NULL, new Looper of a native layer mLooper = new Looper(false); // Equivalent to the Java layer threadlocal.set ()? Looper::setForThread(mLooper); }}Copy the code

MessageQueue of Natvie layer is closely related to the Java layer and the Native layer, but the Looper of the Native layer has no relationship with the Looper of the Java layer

// libutils/Looper.cpp Looper::Looper(bool allowNonCallbacks) : mAllowNonCallbacks(allowNonCallbacks), mSendingMessage(false), mPolling(false), mEpollRebuildRequired(false), MNextRequestSeq (0), mResponseIndex(0), mNextMessageUptime(LLONG_MAX) { To replace the previous version of the pipe structure awakens the event fd mWakeEventFd. Reset (eventfd (0, EFD_NONBLOCK | EFD_CLOEXEC)); LOG_ALWAYS_FATAL_IF(mWakeEventFd.get() < 0, "Could not make wake event fd: %s", strerror(errno)); AutoMutex _l(mLock); // Create epoll handles and initialize rebuildEpollLocked(); }Copy the code

Epoll handle is created and a wake up event handle is added to epoll

//libutils/Looper.cpp void Looper::rebuildEpollLocked() { ...... Epoll_create1 creates an epoll handle instance and registers the wake pipe mepollfd. reset(epoll_create1(EPOLL_CLOEXEC)); . Struct epoll_event eventItem; memset(& eventItem, 0, sizeof(epoll_event)); // zero out unused members of data field union eventItem.events = EPOLLIN; eventItem.data.fd = mWakeEventFd.get(); int result = epoll_ctl(mEpollFd.get(), EPOLL_CTL_ADD, mWakeEventFd.get(), &eventItem); for (size_t i = 0; i < mRequests.size(); i++) { const Request& request = mRequests.valueAt(i); struct epoll_event eventItem; request.initEventItem(&eventItem); // Add the wake up event handle (request.fd) to the ePolled handle (mepollfd.get ()) Add a wake up mechanism for epoll int epollResult = epoll_ctl(mepollfd.get (), EPOLL_CTL_ADD, request.fd, &eventItem); . }}Copy the code

The main contents of the init process of Native layer are as follows:

  1. Initialization of NativeQueueMessage and Looper.
  2. An epoll handle is built to add an epoll event registry to epoll

2.2 Message Reading process

Android Message mechanism 2-Handler(Native layer)

nativePollOnce

//core/jni/android_os_MessageQueue.cpp static void android_os_MessageQueue_nativePollOnce(JNIEnv* env, jobject obj, jlong ptr, Jint timeoutMillis) {// Convert the mPtr passed from the Java layer to nativeMessageQueue * nativeMessageQueue = reinterpret_cast<NativeMessageQueue*>(ptr); NativeMessageQueue ->pollOnce(env, obj, timeoutMillis); }Copy the code

NativeMessageQueue :: pollOnce

//core/jni/android_os_MessageQueue.cpp void NativeMessageQueue::pollOnce(JNIEnv* env, jobject pollObj, int timeoutMillis) { ...... MLooper ->pollOnce(timeoutMillis); . }Copy the code

Stars: Native layer: pollOnce

//libutils/ looper. CPP /** * timeoutMillis: specifies the timeout period * outFd: specifies the file descriptor of the event * outEvents: specifies the event that occurs on the current outFd. EVENT_INPUT includes the following types of events: Readable EVENT_OUTPUT: writable EVENT_ERROR: error EVENT_HANGUP: interrupt *outData: PollOnce (int timeoutMillis, int* outFd, int* outEvents, void** outData) {int result = 0; for (;;) {... Result = pollInner(timeoutMillis); }}Copy the code

Looper::pollInner first calls epoll_wait to enter the blocking topic, and the epollevent added to epoll waits for event occurrence or timeout to trigger the nativeWake() method, which writes characters to eventfd to wake up.

Then collect the event to be processed, and then do the processing. Natvie messages are processed in the following order

  1. Process the Native Message and call the Native Handler to process the Message

  2. Handles events of type Resposne array, POLL_CALLBACK

    //libutils/Looper.cpp

    int Looper::pollInner(int timeoutMillis) {

    . // Poll. int result = POLL_WAKE; mResponses.clear(); mResponseIndex = 0; // We are about to idle. mPolling = true; struct epoll_event eventItems[EPOLL_MAX_EVENTS]; // Wait for an event to occur or timeout to trigger the nativeWake() method, Int eventCount = epoll_wait(mepollfd.get (), eventItems, EPOLL_MAX_EVENTS, timeoutMillis); // No longer idling. mPolling = false; // Acquire lock. mLock.lock(); // Rebuild epoll set if needed. if (mEpollRebuildRequired) { mEpollRebuildRequired = false; rebuildEpollLocked(); goto Done; } // Check for poll error. if (eventCount < 0) { if (errno == EINTR) { goto Done; } an unexpected error: %s", strerror(errno)); result = POLL_ERROR; goto Done; } for (int i = 0; i < eventCount; i++) { int fd = eventItems[i].data.fd; uint32_t epollEvents = eventItems[i].events; If (fd == mwakeEventfd.get ()) {if (fd == mwakeEventfd.get ()) {if (epollEvents & EPOLLIN) {awoken(); } } else { ssize_t requestIndex = mRequests.indexOfKey(fd); if (requestIndex >= 0) { int events = 0; if (epollEvents & EPOLLIN) events |= EVENT_INPUT; if (epollEvents & EPOLLOUT) events |= EVENT_OUTPUT; if (epollEvents & EPOLLERR) events |= EVENT_ERROR; if (epollEvents & EPOLLHUP) events |= EVENT_HANGUP; pushResponse(events, mRequests.valueAt(requestIndex)); }}}Copy the code

    // This is Done, and this is Done

    Done: ; //1. Process Native Message first and call Native Handler to process this Message

    mNextMessageUptime = LLONG_MAX; while (mMessageEnvelopes.size() ! = 0) {... if (messageEnvelope.uptime <= now) { { ...... handler->handleMessage(message); } } else { mNextMessageUptime = messageEnvelope.uptime; break; }}Copy the code

    // Invoke all response callbacks. For (size_t I = 0; i < mResponses.size(); i++) { Response& response = mResponses.editItemAt(i); if (response.request.ident == POLL_CALLBACK) { int fd = response.request.fd; int events = response.events; void* data = response.request.data; int callbackResult = response.request.callback->handleEvent(fd, events, data); } } return result; }

The message retrieval process is as follows

  1. Call NativeMessageQueue and Pollonce in Looper, and finally call Looper’s pollInner
  2. PollInner calls epoll_wait to block, and wakes up when the events added in epoll respond or timeout occurs, etc. _
  3. Execute Native layer Message first, then execute Native layer Resposne array, POLL_CALLBACK type events (such as keystroke events, etc.)
  4. Return to the Java layer and call the Read Message in the Java layer MessageQueue.

2.3 Message Sending Process

Android Message mechanism 2-Handler(Native layer)

nativeWake

//core/jni/android_os_MessageQueue.cpp static void android_os_MessageQueue_nativeWake(JNIEnv* env, jclass clazz, jlong ptr) { NativeMessageQueue* nativeMessageQueue = reinterpret_cast<NativeMessageQueue*>(ptr); NativeMessageQueue ->wake(); }Copy the code

NativeMessageQueue::wake

//core/jni/android_os_MessageQueue. CPP void NativeMessageQueue::wake() { }Copy the code

Looper::wake

Libutils/looper. CPP void Looper::wake() {//write something to mWakeEventfd that will wake up epoll_wait, and the thread will continue to send //Native layer messages. Ssize_t nWrite = TEMP_FAILURE_RETRY(write(mwakeEventFd.get (), &inc, sizeof(Uint64_t)))); . }Copy the code

The message sending process is as follows

  1. EnqueueMessage of the Java layer adds messages to MessageQueue of the Java layer
  2. Call the native method nativeWake if you need to wake up
  3. Call the Wake methods of NativeMessageQueue and Looper in turn
  4. Finally, the wrIT method is called to write something to eventFD to wake up epoll_wait.

This is the end of the Android message mechanism analysis, and finally to the message processing family diagram

Images from: Book: Understanding Android In Depth Volume 3

Third, information

  1. The Android source code
  2. Book: Understanding Android In Depth Volume 3
  3. Android Messaging 2-Handler(Native Layer)
  4. Android Handler mechanism 10 Native implementation
  5. Android MessageQueue nativePollOnce
  6. Epoll summary for IO multiplexing
  7. Select /poll/epoll Comparison analysis
  8. Take a closer look at ThreadLocal memory leaks
  9. Thoroughly understand ThreadLocal and weak references

Four, harvest

  1. Understand the processing flow of message mechanism in Native layer
  2. The wait and wake up of events is implemented by IO multiplexing ePoll and EVENTFD
  3. Each time the Message is obtained, the Message of Native layer is executed first, then the response of Natvice layer addFD is executed, and then the Message in Java layer MessageQueue is executed

Thank you for reading

This is the Android messaging mechanism, next we enter the analysis of Binder learning, welcome to pay attention to the public account “audio and video development journey”, learn and grow together.

Welcome to communicate