takeaway
Original article, reprint please indicate the source.
This article source address: netty-source-code-Analysis
The version of Netty used in this article is version 4.1.6.final: annotated netty source code
The word “Pipeline” translates to “Pipeline”, read here to know the design pattern of students should have thought, here used is “chain of responsibility pattern”. In this article, we will take DefaultChannelPipeline as an example to look at the construction of Pipeline and its important data structures.
1 and other components related to Pipeline
1.1 ChannnelHandler
Handle IO events or intercept IO operations and pass them to the next handler in ChannelPipeline. This is a series of callback methods registered in the chain of responsibility.
Handles an I/O event or intercepts an I/O operation, and forwards it to its next handler in its ChannelPipeline
An I/O event is an “inbound event”, and an I/O operation is an “outbound event”. As I said, I’m not going to call it that, but as I understand it, I’m going to refer to both as “events” and “commands”. Obviously, the meanings of “event” and “operation” are different. “Event” means that an event has occurred and we passively receive it, whereas “operation” means that we actively initiate an action or command.
1.2 ChannelHandlerContext
Each ChannelHandler that is added to ChannelPipeline is wrapped into a ChannelHandlerContext. There are two special ChannelHandlerContext exceptions, HeadContext and TailContext. HeadContext inherits from ChannelInBoundHandler and ChannelOutBoundHandler. TailContext inherits from ChannelInBoundHandler. Each ChannelHandlerContext has two Pointers next and prev, which are used to form a two-way list of ChannelHandlerContext.
2 Pipeline construction method
Let’s take the example of DefaultChannelPipeline and start with its constructor. The Channel is saved in the Pipeline property, and two properties are initialized: succeedFuture and voidPromise. These are two special promises that can be shared, and they’re not the point, so it’s okay if you don’t understand them.
The tail and head are the two special ChannelHandlerContexts that are important components in Pipeline.
protected DefaultChannelPipeline(Channel channel) {
this.channel = ObjectUtil.checkNotNull(channel, "channel");
succeededFuture = new SucceededChannelFuture(channel, null);
voidPromise = new VoidChannelPromise(channel, true);
tail = new TailContext(this);
head = new HeadContext(this);
head.next = tail;
tail.prev = head;
}
Copy the codeThe structure of Pipeline after executing the constructor is shown in the figure below.headandtailMakes up the simplest two-way linked list.
ChannelHandlerContextAt present, onlyHeadContextandTailContext.ChannelHandlerContextIs represented by the narrower rectangle inChannelHandlerBecause ofHeadContextandTailContextIt doesn’t includeChannelHandlerBut inheritanceChannelHandlerThat’s why we have a dotted line here. transversalChannelHandlerSaid isChannelInBoundHandleragainChannelOutBoundHandlerOnly the representation of the top half isChannelInBoundHandlerOnly the representation of the lower half isChannelOutBoundHandler.
3 add ChannelHandler
There are many methods starting with add in ChannelPipeline, these methods are to add ChannelHandler method to ChannelPipeline.
addAfterTo a:ChannelHandlerAdd backaddBeforeTo a:ChannelHandlerAdd in front of theaddFirst: added to the header, not inheadIn front, but right next to each otherheadIn theheadThe back of theaddLast: added to the tail, not intailBehind, but right next totailIn thetailIn the front of the
We take the most commonly used addLast method as an example to analyze Pipeline to add ChannelHandler operations. The addLast method posted here is one we’ve already seen in the “Server startup Process” article. EventExecutorGroup in the method argument means that we can set Excutor separately for this ChannelHandler without using the EventLoop bound to the Channel. Normally we don’t do this, so the group argument is null.
We wrap ChannelHandler as a ChannelHandlerContext, add it to the tail, and then call the HandlerAdded method of ChannelHandler.
One problem with calling the HandlerAdded method is that adding the ChannelHandler does not need to happen in the EventLoop thread, whereas the HandlerAdded method must. Newctx.setaddpending () is called to set the current HandlerContext to ADD_PENDING. And invoke callHandlerCallbackLater (newCtx, true) will be an asynchronous task is added to a one-way team list, namely pendingHandlerCallbackHead the linked list.
If EventLoop is currently bound, then see if the calling thread is an EventLoop thread. If not, then submit an asynchronous task to EventLoop to call callHandlerAdded0. Otherwise, call callHandlerAdded0 directly.
Let’s examine newContext, callHandlerCallbackLater and callHandlerAdd0 in sequence.
@Override
public final ChannelPipeline addLast(EventExecutorGroup group, String name, ChannelHandler handler) {
final AbstractChannelHandlerContext newCtx;
synchronized (this) {
// Wrap 'handler' into 'HandlerContext' first
newCtx = newContext(group, filterName(name, handler), handler);
// Add to the tail
addLast0(newCtx);
// If 'EventLoop' is not already bound, the 'HandlerAdded' method will be called later.
if(! registered) { newCtx.setAddPending(); callHandlerCallbackLater(newCtx,true);
return this;
}
// If an EventLoop is bound and the current thread is not an EventLoop thread, submit an asynchronous task and initiate an asynchronous task to call the HandlerAdded method.
EventExecutor executor = newCtx.executor();
if(! executor.inEventLoop()) { newCtx.setAddPending(); executor.execute(new Runnable() {
@Override
public void run(a) { callHandlerAdded0(newCtx); }});return this; }}// If the current thread is an EventLoop thread, the HandlerAdded method is called directly.
callHandlerAdded0(newCtx);
return this;
}
Copy the code3.1 newContext
First it seems the newContext method, direct call here DefaultChannelHandlerContext constructor, let’s look inside.
private AbstractChannelHandlerContext newContext(EventExecutorGroup group, String name, ChannelHandler handler) {
return new DefaultChannelHandlerContext(this, childExecutor(group), name, handler);
}
Copy the codeIn the constructor of DefaultChannelHandlerContext again call the parent class AbstractChannelHandlerContext constructor, preserved the handler attributes. The isInboud and isOutbound methods are called before the parent constructor is called to determine whether the current Handler is ChannelInBoundHandler or ChannelOutBoundHandler. These two methods are simple and do not expand any more.
DefaultChannelHandlerContext(
DefaultChannelPipeline pipeline, EventExecutor executor, String name, ChannelHandler handler) {
super(pipeline, executor, name, isInbound(handler), isOutbound(handler));
if (handler == null) {
throw new NullPointerException("handler");
}
this.handler = handler;
}
Copy the codeSee next AbstractChannelHandlerContext constructor, here is very simple, save for a few properties, let’s look at the ordered this property. Ordered: whether an EventExecutor executes an asynchronous task in the order in which it was added. Normally, null executor indicates the EventLoop thread bound to the Channel. And EventLoop threads are all implementations of OrderedEventExecutor. So here we don’t consider the case where ordered is false.
AbstractChannelHandlerContext(DefaultChannelPipeline pipeline, EventExecutor executor, String name,
boolean inbound, boolean outbound) {
this.name = ObjectUtil.checkNotNull(name, "name");
this.pipeline = pipeline;
this.executor = executor;
this.inbound = inbound;
this.outbound = outbound;
// Its ordered if its driven by the EventLoop or the given Executor is an instanceof OrderedEventExecutor.
ordered = executor == null || executor instanceof OrderedEventExecutor;
}
Copy the codeWe mentioned that ChannelHandlerContext can specify an EventExecutor separately in the constructor, or if not, use the EventLoop to which the Channel is bound. Where is the code? In AbstractChannelHandlerContext# executor method is very simple, if not for the current ChannelHandler specified excutor return Channel binding EventLoop.
@Override
public EventExecutor executor(a) {
if (executor == null) {
return channel().eventLoop();
} else {
returnexecutor; }}Copy the code3.2 callHandlerCallbackLater
The handlerAdded method of the ChannledHandler is called after the ChannelHandler is added, but it may not be bound to an EventLoop, and the call to the handlerAdded method must be executed within the EventLoop thread. Right now you need to call callHandlerCallbackLater method in add a PendingHandlerAddedTask pendingHandlerCallbackHead list.
private void callHandlerCallbackLater(AbstractChannelHandlerContext ctx, boolean added) {
assert! registered; PendingHandlerCallback task = added ?new PendingHandlerAddedTask(ctx) : new PendingHandlerRemovedTask(ctx);
PendingHandlerCallback pending = pendingHandlerCallbackHead;
if (pending == null) {
pendingHandlerCallbackHead = task;
} else {
// Find the tail of the linked-list.
while(pending.next ! =null) { pending = pending.next; } pending.next = task; }}Copy the codeNext, let’s look at the code for the PendingHandlerAddedTask. The logic is in the execute method, which calls callHandlerAdded0 directly.
private final class PendingHandlerAddedTask extends PendingHandlerCallback {
PendingHandlerAddedTask(AbstractChannelHandlerContext ctx) {
super(ctx);
}
@Override
public void run(a) {
callHandlerAdded0(ctx);
}
@Override
void execute(a) {
EventExecutor executor = ctx.executor();
if (executor.inEventLoop()) {
callHandlerAdded0(ctx);
} else {
try {
executor.execute(this);
} catch (RejectedExecutionException e) {
}
}
}
}
Copy the code3.3 callHandlerAdded0
The callHandlerAdded0 method will eventually be called, whether the handlerAdded is called late without an EventLoop or immediately if an EventLoop is already attached. Two things are done here. One is to call the handlerAdded method of ChannelHandler, and the other is to set the state of the HandlerContext to ADD_COMPLETE.
private void callHandlerAdded0(final AbstractChannelHandlerContext ctx) {
try {
ctx.handler().handlerAdded(ctx);
ctx.setAddComplete();
} catch (Throwable t) {
}
Copy the code3.4 Pipeline After Multiple ChannelHandlers are added
Remember our “Netty overall architecture diagram”? Here we putPipelinePart of the separate zoom out to take a look, after adding moreChannelHandlerAfter that,PipelineThe structure of theta is this.
4 remove ChannelHandler
In Pipeline, there are several methods beginning with remove. These methods are used to remove ChannelHandler.
remove(ChannelHandler handler)From:headtotailLook for,= =Determine whether it is the same instance and delete only the first one.remove(Class<T> handlerType)From:headtotailLook for,isAssignableFromMethod to determine whether it is a qualified type and delete only the first one.remove(String name)From:headtotailLook for,nameFor exact match search, remove only the first one becausenameIt can’t be repeated, so I’m going to delete the first one and the only one.removeFirst: deleteheadThe latter one cannot be deletedtail.removeLast: deletetailCannot be deletedhead.
The way either delete after the search to the corresponding HandlerContext will be called to remove (final AbstractChannelHandlerContext CTX) method, search process is simple, we no longer, Directly see the remove (final AbstractChannelHandlerContext CTX) method.
Let’s see if the implementation of this method is similar to addLast(EventExecutorGroup Group, String name, ChannelHandler handler), very similar. First remove the ChannelHandlerContext from the bidirectional list, then call callHandlerRemoved0, which calls the handlerRemoved method, This call must be made within the EventLoop thread. If you haven’t binding when deleting EventLoop is an asynchronous task is added to the list pendingHandlerCallbackHead.
Submit an asynchronous task to EventLoop if an EventLoop is bound and the current thread is not an EventLoop thread, otherwise call callHandlerRemoved0 directly.
private AbstractChannelHandlerContext remove(final AbstractChannelHandlerContext ctx) {
synchronized (this) {
// Remove 'ChannelHandlerContext' from the two-way list
remove0(ctx);
// If 'EventLoop' is not already bound, the 'handlerRemoved' method is called later
if(! registered) { callHandlerCallbackLater(ctx,false);
return ctx;
}
// If 'EventLoop' has been bound, but the current thread is not an 'EventLoop' thread, then an asynchronous task is initiated calling callHandlerRemoved0
EventExecutor executor = ctx.executor();
if(! executor.inEventLoop()) { executor.execute(new Runnable() {
@Override
public void run(a) { callHandlerRemoved0(ctx); }});returnctx; }}// If the current thread is an 'EventLoop' thread, call callHandlerRemoved0 directly.
callHandlerRemoved0(ctx);
return ctx;
}
Copy the codeCallHandlerCallbackLater approach our front has been analyzed, and the difference is that when adding ChannelHandler to list to add here is that the PendingHandlerRemovedTask, this class is also very simple, no longer.
Let’s just look at the callHandlerRemoved0 method here. The method is simple: call the handlerRemoved method, and set the state of the ChannelHandlerContext to REMOVE_COMPLETE.
private void callHandlerRemoved0(final AbstractChannelHandlerContext ctx) { // Notify the complete removal. try { try { ctx.handler().handlerRemoved(ctx); } finally { ctx.setRemoved(); } } catch (Throwable t) { fireExceptionCaught(new ChannelPipelineException( ctx.handler().getClass().getName() + ".handlerRemoved() has thrown an exception.", t)); }}Copy the code4 pendingHandlerCallbackHead task in the list when calling
In AbstractUnsafe register0 method, after binding EventLoop invokes the pipeline. The invokeHandlerAddedIfNeeded () method, We have a look at the pipeline. InvokeHandlerAddedIfNeeded () method.
private void register0(ChannelPromise promise) {
try {
/ / to do those in the binding EventLoop trigger to add handler before the operation, the operation is in the pendingHandlerCallbackHead in pipeline, a linked list
pipeline.invokeHandlerAddedIfNeeded();
}
Copy the codeInvokeHandlerAddedIfNeeded method call callHandlerAddedForAllHandlers method, let’s watch it then.
final void invokeHandlerAddedIfNeeded(a) {
assert channel.eventLoop().inEventLoop();
if (firstRegistration) {
firstRegistration = false; callHandlerAddedForAllHandlers(); }}Copy the codeCallHandlerAddedForAllHandlers method of logic za spread is no longer, is very simple, is to traverse pendingHandlerCallbackHead the singly linked list, each element in turn call the execute method, and clear the singly linked list.
private void callHandlerAddedForAllHandlers(a) {
final PendingHandlerCallback pendingHandlerCallbackHead;
synchronized (this) {
assert! registered; registered =true;
pendingHandlerCallbackHead = this.pendingHandlerCallbackHead;
this.pendingHandlerCallbackHead = null;
}
PendingHandlerCallback task = pendingHandlerCallbackHead;
while(task ! =null) { task.execute(); task = task.next; }}Copy the code5 concludes
Pipeline the most important data structure is composed of multiple ChannelHandlerContext two-way linked list, and each ChannelHandlerContext contains a ChannelHandler, ChannelHandler can be added or removed. There are two special ChannelHandlerContext in Pipeline, namely HeadContext and TailContext. These two ChannelHandlerContext do not contain ChannelHandler, but adopt inheritance method. HeadContext implements ChannelOutBoundHandler and ChannelInBoundHandler, while TailContext implements ChannelInBoundHandler.
About the author
Wang Jianxin, senior Java engineer of architecture department, mainly responsible for service governance, RPC framework, distributed call tracking, monitoring system, etc. Love technology, love learning, welcome to contact and exchange.
Original articles, code words are not easy to share, like the hand has lingering fragrance.