Let’s cut back on the foreplay this time, get right to the point, and we’ll get started.
The opening question is all about blowing. (This should be funny)
// File name: index.js
// We tried to simulate all asynchronous scenarios, including timers, Promise, nextTick, etc
setTimeout((a)= > {
console.log('timeout 1');
}, 1);
process.nextTick((a)= > {
console.log('nextTick 1');
});
fs.readFile('./index.js', (err, data) => {
if(err) return;
console.log('I/O callback');
process.nextTick((a)= > {
console.log('nextTick 2');
});
});
setImmediate((a)= > {
console.log('immediate 1');
process.nextTick((a)= > {
console.log('nextTick 3');
});
});
setTimeout((a)= > {
console.log('timeout 2');
process.nextTick((a)= > {
console.log('nextTick 4');
});
}, 100);
new Promise((resolve, reject) = > {
console.log('promise run');
process.nextTick((a)= > {
console.log('nextTick 5');
});
resolve('promise then');
setImmediate((a)= > {
console.log('immediate 2');
});
}).then(res= > {
console.log(res);
});
Copy the codeNote: The above code is executed in Node V10.7.0. There are some differences between the browser event loop and Node. If you are interested, you can find some information for yourself.
Ok, so the code above refers to timers, nextTick, Promise, setImmediate, and I/O operations. Scalp is a bit small numb ha, did everybody think good answer? Check it out!
promise run
nextTick 1
nextTick 5
promise then
timeout 1
immediate 1
immediate 2
nextTick 3
I/O callback
nextTick 2
timeout 2
nextTick 4
Copy the codeHow’s that？ Is it what you think it is? If it’s not, just listen to me.
event loop
In Node.js, the Event loop is based on Libuv. By looking at the documentation of Libuv, it can be found that the entire Event loop is divided into six stages:
- Timers: Timers related tasks. Node is concerned with executing the expiration callbacks of setTimeout() and setInterval()
- Pending Callbacks: Callbacks that perform certain system operations
- Idle, prepare: internal use
- Poll: Performs I/O callback, which under certain conditions blocks at this stage
- Check: Executes the setImmediate callback
- Close callbacks: If the socket or Handle is closed, a close event is triggered at this stage and the callback for the close event is executed
┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ > │ timers │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ │ pending Callbacks │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ │ idle, Prepare │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ │ incoming: │ │ │ poll │ < ─ ─ ─ ─ ─ ┤ connections, │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ data, Etc. │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ │ check │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ └ ─ ─ ┤ close callbacks │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘Copy the codeThe event loop code is in the file deps/uv/ SRC/Unix /core.c.
int uv_run(uv_loop_t* loop, uv_run_mode mode) {
int timeout;
int r;
int ran_pending;
// Determine whether the event loop continues
r = uv__loop_alive(loop);
if(! r) uv__update_time(loop);while(r ! =0 && loop->stop_flag == 0) {
uv__update_time(loop); // Update time
uv__run_timers(loop); / / timers
ran_pending = uv__run_pending(loop); // Pending Callbacks
uv__run_idle(loop); / / idle period
uv__run_prepare(loop); / / the prepare phase
timeout = 0;
// Set the timeout period for the poll phase. In the following cases, the timeout period is set to 0. In this case, the poll does not block
// 1. Stop_flag is not 0
// 2. No active handles or request
// 3. The handle queue in idle, pending callback, or close phases is not empty
// Otherwise, the timeout is set to the time of the nearest timer
if((mode == UV_RUN_ONCE && ! ran_pending) || mode == UV_RUN_DEFAULT) timeout = uv_backend_timeout(loop);/ / poll phase
uv__io_poll(loop, timeout);
/ / check phase
uv__run_check(loop);
/ / close stage
uv__run_closing_handles(loop);
if (mode == UV_RUN_ONCE) {
uv__update_time(loop);
uv__run_timers(loop);
}
r = uv__loop_alive(loop);
if (mode == UV_RUN_ONCE || mode == UV_RUN_NOWAIT)
break;
}
if(loop->stop_flag ! =0)
loop->stop_flag = 0;
return r;
}
Copy the codeRegistration plus trigger
In this section we’ll focus on how Node registers and executes the timer we wrote in the Event loop.
Taking setTimeout as an example, we first went to the timers.js file and found the setTimeout function. We mainly focused on the following two sentences:
function setTimeout(callback, after, arg1, arg2, arg3) {
// ...
const timeout = new Timeout(callback, after, args, false);
active(timeout);
return timeout;
}
Copy the codeWe see that it has a new Timeout class, and we follow this lead to the constructor of Timeout:
function Timeout(callback, after, args, isRepeat) {
// ...
this._onTimeout = callback;
// ...
}
Copy the codeWe’ll focus on this one, where Node attaches the callback to the _onTimeout property. So when is this callback executed? If we do a global search for _onTimeout(), we can see that a method called ontimeout executes the callback. Ok, let’s follow the trail. You can find the call path processTimers -> listOnTimeout -> tryOnTimeout -> onTimeout -> _onTimeout.
Finally, at the end of the file, we find these lines of code:
const {
getLibuvNow,
setupTimers,
scheduleTimer,
toggleTimerRef,
immediateInfo,
toggleImmediateRef
} = internalBinding('timers');
setupTimers(processImmediate, processTimers);
Copy the codeSetupTimers are retrieved from internalBinding(‘timers’). If we look at internalBinding, we see that this is where the JS code is associated with the built-in module. So we followed the lead and went to the SRC directory to find the timers file. Sure enough, we found a file called timers.cc and a function called SetupTimers.
void SetupTimers(const FunctionCallbackInfo<Value>& args) {
CHECK(args[0]->IsFunction());
CHECK(args[1]->IsFunction());
auto env = Environment::GetCurrent(args);
env->set_immediate_callback_function(args[0].As<Function>());
env->set_timers_callback_function(args[1].As<Function>());
}
Copy the codeThe above args[1] is the processTimers we passed in. In this function, we have registered processTimers successfully with node.
How does that trigger a callback? Here we first look at the timers function in the Event loop code and follow along:
void uv__run_timers(uv_loop_t* loop) {
struct heap_node* heap_node;
uv_timer_t* handle;
for (;;) {
heap_node = heap_min(timer_heap(loop));
if (heap_node == NULL)
break;
handle = container_of(heap_node, uv_timer_t, heap_node);
if (handle->timeout > loop->time)
break; uv_timer_stop(handle); uv_timer_again(handle); handle->timer_cb(handle); }}Copy the codeLet’s look at the handle->timer_cb(handle) line. Where is this function defined? We searched timer_cb globally and found the following line in uv_timer_start:
handle->timer_cb = cb;
Copy the codeSo we know that uv_timer_start was called to mount the callback function to Handle. So what is cb? Cb is timers_callback_function. It looks familiar, doesn’t it? This is the function we registered above to trigger the callback, processTimers.
Uv_timer_start = uv_timer_start = uv_timer_start; The ScheduleTimer function is called by internalBinding. This function is called by internalBinding. This function is called by internalBinding.
It’s a little different here. The latest tag version is now different from the latest node code on Github. In a pr refactoring of timer_wrap.cc to timers.cc and removing the TimerWrap class, First, add the data structure corresponding to timer:
// This is in the version with TimeWrap
// The corresponding time is followed by a timer list
refedLists = {
1000: TimeWrap._list(TimersList(item<->item<->item<->item)),
2000: TimeWrap._list(TimersList(item<->item<->item<->item)),
};
// This is the scheduleTimer version
refedLists = {
1000: TimersList(item<->item<->item<->item),
2000: TimersList(item<->item<->item<->item),
};
Copy the codeIn the TimeWrap version, JS calls uv_timer_start by calling the instantiated start() function.
The scheduleTimer version registers the timerList to the Uv_timer by comparing which timer is the most recently executed.
So why the change? The Immediate has its own ImmediateQueue, which is also a linked list. The Immediate should have a single UV_TIMer_T Handle stored in the Environment.
Here is only one timer, the rest of you to have a look, follow this idea you will certainly gain.
Event loop flow
When loading node, register setTimeout and setInterval callbacks to timerList and Promise. Resolve callbacks to microTasks. Register setImmediate into its immediateQueue and Process. nextTick into its nextTickQueue.
When we start the Event loop, we first enter the Timers phase (we only look at the related phases above) and then determine whether the timerList is out of time. If it is, we execute it and do not proceed to the next phase (actually nextTick, more on that later).
Next we move to the poll phase, where we calculate how long we need to block polling in this phase (simply the time of the next timer) and wait for the listening event.
The next phase is the Check phase, corresponding to the immediate immediateQueue, and then the poll phase will be skipped directly to the Check phase to perform the setImmediate callback.
Where are nextTick and microTasks? Don’t panic. Just listen to me.
Process. NextTick and microTasks
Now that we have the experience of finding the timer, let’s go ahead and see how nextTick is executed.
After scouring, we can find a function called _tickCallback, which continuously fetches the nextTick callback execution from nextTickQueue.
function _tickCallback() {
let tock;
do {
while (tock = queue.shift()) {
// ...
const callback = tock.callback;
if (tock.args === undefined)
callback();
else
Reflect.apply(callback, undefined, tock.args);
emitAfter(asyncId);
}
tickInfo[kHasScheduled] = 0;
runMicrotasks();
} while(! queue.isEmpty() || emitPromiseRejectionWarnings()); tickInfo[kHasPromiseRejections] =0;
}
Copy the codeWhat do we see? After executing the nextTick callback, it executes runMicrotasks. It’s all clear that microTasks are executed after all nextTick callbacks have been executed. When was nextTick implemented?
BOOTSTRAP_METHOD(_setupNextTick, SetupNextTick) is found in bootstrap. cc, so we need to find the SetupNextTick function.
void SetupNextTick(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
// ...
env->set_tick_callback_function(args[0].As<Function>());
// ...
}
Copy the codeWe register __tickCallback with node and call this function via tick_callback_function in C++.
NextTixk execution is triggered when InternalCallbackScope calls Close.
void InternalCallbackScope::Close() { if (closed_) return; closed_ = true; HandleScope handle_scope(env_->isolate()); / /... if (! tick_info->has_scheduled()) { env_->isolate()->RunMicrotasks(); } / /... if (! tick_info->has_scheduled() && ! tick_info->has_promise_rejections()) { return; } / /... if (env_->tick_callback_function()->Call(process, 0, nullptr).IsEmpty()) { failed_ = true; }}Copy the codeNextTick: RunMicrotasks nextTick: RunMicrotasks This is an optimization for event loop. If process.nextTick is not present, RunMicrotasks are called directly from node to speed things up.
Now in Node. cc we have a place to call Close:
MaybeLocal<Value> InternalMakeCallback(Environment* env,
Local<Object> recv,
const Local<Function> callback,
intargc, Local<Value> argv[], async_context asyncContext) { CHECK(! recv.IsEmpty());InternalCallbackScope scope(env, recv, asyncContext);
scope.Close();
return ret;
}
Copy the codeInternalMakeCallback() is called in async_wrap.cc AsyncWrap::MakeCallback().
SetImmediate Registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate registers setImmediate. TimeWrap extends AsyncWrap and calls MakeCallback() when executing a callback. Now remove TimeWrap. We’ll go to the js code and find something like this:
const { _tickCallback: runNextTicks } = process;
function processTimers(now) {
runNextTicks();
}
Copy the codeAll clear, _tickCallback is executed here after removing TimeWrap, so we just missed it in C++.
In fact, _tickCallback is executed after each stage, but in a slightly different way.
The answer to parse
Now that you know a little bit about Event Loops, let’s look at the code at the beginning of this article.
The first step
First run the code in Promise, which prints Promise Run, and Promise. Resolve puts then into microTasks.
The second step
One thing to mention here is that after nextTick is registered, the SetupNextTick function is run after bootstrap is built. At this point, the nextTickQueue and MicroTasks are cleared. So output nextTick 1, nextTick 5, promise then.
The third step
After bootstrap, the event loop is displayed. The first stage is timers. At this time, the timeout 1 timer expires and the callback output timeout 1 is executed. The nextTickQueue and MicroTasks are empty. If there is no task, proceed to the next phase, and there is immediate, so skip the poll, go to check, and execute the immediate callback. ImmediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue immediateQueue Clear the nextTickQueue and MicroTasks output nextTick 3.
The fourth step
In this round, after the file is read and timers are not expired, the poll phase is carried out, timeout is set to timeout 2, callback output I/O callback is executed, and nextTick 2 is pushed to nextTickQueue. If there are no other I/O events during blocking, clear nextTickQueue and MicroTasks and print nextTick 2.
Step 5
At this time, it is time for timers. Execute the callback of timeout 2 and print timeout 2 to push nextTick 4 to nextTickQueue. At this time, there is no timer in the timeList. Clear nextTickQueue and MicroTasks output nextTick 4.
conclusion
I don’t know if you understand, the whole process is still relatively rough, in the process of learning also saw a lot of source analysis, but Node development is very fast, a lot of analysis has been outdated, the source code changed a lot, but for clear thinking is still very useful.
If you feel OK, OK, a little useful, welcome to my GitHub to give a little star, I will be very comfortable, haha.
Wen/Xiao Xuan
The secret of seamless is: Are you happy doing technology?
Sound/fluorspar
Editor’s note: the author is also playing nuggets, pay attention to him
This article has been authorized by the author, the copyright belongs to chuangyu front. Welcome to indicate the source of this article. Link to this article: knownsec-fed.com/2018-09-13-…
To subscribe for more sharing from the front line of KnownsecFED development, please search our wechat official account KnownsecFED. Welcome to leave a comment to discuss, we will reply as far as possible.
Thank you for reading.