Abstract: The MapReduceService cluster uses NTP for clock synchronization. This document describes the NTP mechanism of the MapReduce Service cluster and how to configure NTP.

This article is shared with The MapReduce Service Replacing an External Clock Source for a Cluster in Huawei Cloud community by Tangyuxiaobao29.

1. MapReduceService Cluster NTP mechanism

1. The active OMS node of FusionInsight synchronizes data with the NTP external clock source

2. Synchronize all service nodes to the active OMS node

Based on the preceding principles, the entire cluster is finally synchronized with the NTP clock source. The biggest advantage of this method is that the clocks in the entire cluster are synchronized with the active OMS at any time, ensuring that the clocks of the nodes in the cluster are synchronized. If there is no problem with the external NTP clock source, then the entire cluster is synchronized with the external NTP clock source!

2. MapReduceService Configuration description of the NTP server

1. Install FusionInsight Manager. Configure the NTP server

In the FusionInsightManager installation configuration file install.ini, use ntp_server_IP to set the IP address of the external NTP clock server. This parameter is optional.

If this parameter is not specified, the active management node serves as the NTP server by default. NTP status of the active OMS node after installation

NTP status of non-active OMS nodes in the cluster

If this parameter is set and multiple NTP servers exist, use commas (,) to separate IP addresses, for example, 192.168.34.185. NTP status of the active OMS node after installation

The NTP status of non-active OMS nodes in the cluster remains unchanged

2. Replace the cluster NTP server after installation

1. Operation scenarios

After FusionInsight Manager is installed, if no NTP server is configured or the configured NTP server is no longer used, you can specify or replace a new NTP server for the cluster so that the cluster can synchronize time with the new NTP clock source.

2. Impact on the system

Replacing an NTP server is a high-risk operation. The cluster time may change after the replacement. If the time difference between the NTP server and the cluster is greater than 150 seconds before replacement, stop the cluster to prevent data loss. Services cannot be accessed during the cluster shutdown.

3. Prerequisites

A new NTP server has been prepared, obtained its IP address, and configured the network between the cluster and the new NTP server. Ensure that the NTP service on the server is running properly. Otherwise, the operation fails. An IPv4 cluster supports only an external clock source on an IPv4 network. An IPv6 cluster supports only an external clock source on an IPv6 network. If the cluster uses IPv4 mode, the external clock source can use only IPv4 addresses. If the cluster uses IPv6 mode, the external clock source can use only IPv6 addresses.

4. Operation steps

1. Log in to FusionInsight Manager to check whether uncleared alarms exist

  • If yes, clear the alarm by referring to Fault Management. After the alarm is cleared, go to 2.

  • If no, go to 2.

2. Log in to the active and standby management nodes as user omm.

3. Run the following command on the active management node to view the gateway address of the management plane.

cat ${BIGDATA_HOME}/om-server/OMS/workspace/conf/oms-config.ini| grep om_gateway

4. Ping the management plane gateway on the active and standby management nodes to check whether the management plane gateway can be pinged from the active and standby management nodes

  • If yes, go to 5.

  • If no, contact the network administrator to rectify the network fault. Then, go to 5.

5. Run the following command on the active management node to check the time difference between the cluster and the new NTP server. The unit is second.

For example, to check the time difference with NTP server 192.168.34.185, run the ntpdate -d 192.168.34.185 command.

0.034136 indicates the time deviation. A positive value indicates that the NTP server time is faster than the current cluster time. A negative value indicates that the NTP server time is slower than the current cluster time.

6. Check whether the absolute value of the time deviation exceeds 150S.

  • If yes, go to 7.

  • If no, the absolute value of the time difference cannot exceed 150. Go to 9 as user omm.

7. Check whether the cluster can be stopped.

  • If yes, stop upper-layer services and the cluster, and go to 8.

  • If no, no further action is required.

8. Check whether the NTP server time is slower than the current cluster time

  • If yes, the NTP server time is slow. If operation Succeeded is displayed on the stop cluster page, wait for an interval of about 5 and go to 10 as user omm.

  • If no, the NTP server time is fast. If Operation Succeeded is displayed on the stop cluster page, go to 10 as user omm.

9. Run the following command on the active management node to replace the NTP server.

Sh ${BIGDATA_HOME} / om – server/om/bin/tools/modifyntp sh — ntp_server_ip192. 168.34.185

Then check the NTP status

10. Run the following command on the active management node to forcibly synchronize time with the NTP server 192.168.34.185 and replace the NTP server. No further action is required.

Sh ${BIGDATA_HOME} / om – server/om/bin/tools/modifyntp sh – ntp_server_ip 192.168.34.185 – force_sync_time

instructions

  • Do not set the NTP server IP address to the IP address of a node in the cluster. Otherwise, the node may be disconnected from the service network of the active and standby OMS nodes.

  • Currently, a maximum of two NTP servers can be configured to provide services in active/standby mode. Use commas (,) to separate the NTP servers

  • If the cluster is stopped, restart the cluster after replacing the NTP server.

  • If the forcible time synchronization command is not executed, the system synchronizes time at the rate of 200ms/h.

  • After forcibly synchronizing time is executed, it takes about 5 minutes for cluster nodes to synchronize time.

Click to follow, the first time to learn about Huawei cloud fresh technology ~