This is the 21st day of my participation in Gwen Challenge. For more information, see: Gwen Challenge 😄
Application, distribution and execution of tasks
TaskTracker. Run () connects to JobTracker. The TaskTracker startup process initializes a set of parameters and services (covered in a separate section), An attempt is then made to connect to the JobTracker service (that is, the InterTrackerProtocol interface must be implemented), and if the connection is broken, a loop is made to try to connect to JobTracker and reinitialize all members and parameters, as described in the run() method.
The main loop TaskTracker. OfferService ()
If the connection to the JobTracker service is successful, The TaskTracker calls the offerService() function to enter the main execution loop. This loop communicates with JobTracker every 10 seconds, calling transmitHeartBeat() to get a HeartbeatResponse. Then call the getActions() function of the HeartbeatResponse to get all the instructions sent by JobTracker, which is an array of TaskTrackerActions. If it’s a new task directive, LaunchTaskAction, then startNewTask() is called to execute the new task,
If it is CommitTaskAction
Otherwise, join the tasksToCleanup queue and hand it over to a taskCleanupThread, such as KillJobAction or KillTaskAction.
TaskTracker. TransmitHeartBeat () to obtain the JobTracker instructions
In transmitHeartBeat() processing, TaskTracker creates a new TaskTrackerStatus object to record the current task status, It is then sent through the IPC interface by calling JobTracker’s heartbeat() method and receives the new instruction, which returns the TaskTrackerAction array. Before this call, TaskTracker checks the number of tasks currently executing, the space usage of the local disk, etc., and sets heartbeat()’s askForNewTask to true if new tasks can be received. After the operation succeeds, update related statistics.
JobTracker Scheduling Jobs Step 2: Dispatch tasks
When JobTracker receives a TaskTracker heartbeat() call, it first checks to see if the last heartbeat response is complete, if there is no request to start or restart the task, and if all is well, it processes the heartbeat. JobTracker uses its scheduler taskScheduler to assemble tasks into a task list. The assignTasks() method of the taskScheduler is implemented. Once you have the data for tasks, encapsulate them in lanuchTaskActions and send them back to The TaskTracker to perform the task. At this point, JobTracker’s Hearbeat () stops sending tasks.
Here’s a quick look at how JobQueueTaskScheduler implements assignTasks(). First, it checks to see how many map and Reduce tasks TaskTracker can still do, whether the number of tasks to be dispatched exceeds this number, and whether it exceeds the average number of tasks available for the cluster. If none is exceeded, assign a MapTask or ReduceTask to the TaskTracker. To generate a Map task, use JobInProgress’s obtainNewMapTask() method, In essence, findNewMapTask() of JobInProgress calls nonRunningMapCache to get TaskInProgress of the Map task; And Reduce task using JobInProgress. ObtainNewReduceTask () method, In essence, findNewReduceTask() of JobInProgress was finally called to access nonRuningReduceCache to obtain TaskInProgress of the Reduce task.
Thank you for reading this, if this article is well written and if you feel there is something to it
Ask for a thumbs up 👍 ask for attention ❤️ ask for share 👥 for 8 abs I really very useful!!
If there are any mistakes in this blog, please comment, thank you very much! ❤ ️ ❤ ️ ❤ ️ ❤ ️