White Blacklist Configuration

Whitelist: Whitelisted host IP addresses can be used to store data. Enterprise: Whitelist is configured to prevent malicious access attacks by hackers. Blacklist: Indicates that the host IP address in the blacklist cannot be used to store data. Enterprise: Configure a blacklist to decommission servers.

  1. Create whitelist and blacklist files in the /opt/module/hadoop-3.1.3/etc/hadoop directory on the NameNode node
  2. Add the dfs.hosts configuration parameter to the hdFS-site. XML configuration file
  3. Distribute the configuration file whitelist, hdFS-site.xml
  4. The first time you add a whitelist, you must restart the cluster. You only need to refresh the NameNode
  5. Refresh command: HDFS dfsadmin -refreshnodes

Set the following parameters:

<! - white list - > < property > < name > DFS. Hosts < / name > < value > / opt/module/hadoop - 3.1.3 / etc/hadoop/whitelist < value > / < / property > <! - black - > < property > < name > DFS. Hosts. Exclude < / name > < value > / opt/module/hadoop - 3.1.3 / etc/hadoop/blacklist value > < / </property>Copy the code

New nodes in service:

  1. Prepare the environment (each module configuration);
  2. Directly start DataNode to associate with cluster:
    • hdfs –daemon start datanode
    • yarn –daemon start nodemanager
  3. Add new servers to the whitelist
  4. Distribute the configuration file.
  5. Refresh the NameNode
    • hdfs dfsadmin -refreshNodes

Retired node

  1. Add retired nodes to the blacklist.
  2. Distribution configuration;
  3. Refresh the NameNode
    • hdfs dfsadmin -refreshNodes
  4. If the decommissioned node is in the decommission in Progress state, the data node is replicating blocks to other nodes. Stop the decommissioned node and the node explorer until the decommissioned node is decommissioned. Note: If the number of copies is 3 and the number of nodes in service is less than or equal to 3, it cannot be retired successfully. You need to modify the number of copies before retiring.