1. Prepare the installation environment

1. Prepare the installation package

You need to download jdK-8u144-linux-x64.tar. gz, ambari-xx-centos7.tar.gz, HDP-XX-Centos7-rpm.tar. gz, Hdp-utils-xx-centos7.tar. gz to local (RPM package is very large), link:

http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.5/ambari-2.6.1.5-centos7.tar.gzhttps://download.cs dn.net/download/ljk168/10351315http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0/HDP-GPL-2.6.4.0- centos7-rpm.tar.gzhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.g zhttp://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/HDP-2.6.4.0-centos7-rpm.tar.gzCopy the code

2. Prepare the host.

Modify vm network Settings, disable SElinux, change the host name, disable the firewall, secure Shell (SSH) security exemption, enable the HTTP service, and configure clock synchronization. The network requirements are as follows: In NAT mode, all hosts are on the same network segment and can access the Internet. Modify the IP address, gateway, and DNS.

vim /etc/sysconfig/network-scripts/ifcfg-ens33
Copy the code

Copy the code

Modified ifcfG-EN33 DHCP automatic allocation to static addressing, and added IP/mask/gateway configuration. (No matter how you configure the gateway, do not configure 192.168.x.1 because IP number 1 is the IP address of VMnet8.)

TYPE="Ethernet"PROXY_METHOD="none"BROWSER_ONLY="no"BOOTPROTO="static"DEFROUTE="yes"IPV4_FAILURE_FATAL="no"IPV6INIT="yes" IPV6_AUTOCONF="yes"IPV6_DEFROUTE="yes"IPV6_FAILURE_FATAL="no"IPV6_ADDR_GEN_MODE="stable-privacy"NAME="ens33"UUID="5ed941 C2 - ae - a76d deb4-48-838503 c8683c "DEVICE =" ens33 "ONBOOT =" yes "IPADDR =" 192.168.242.111 "GATWAY =" 192.168.242.2 NETMASK = 255.255. 255.0 "DNS1 =" 192.168.242.2"Copy the code

After the configuration is complete, run the service network restart command to restart the nic Settings. If the network cannot connect to the Internet (ping www.baidu.com), you can add the default DNS.

Route add default gw 192.168.242.2Copy the code

Copy the code

Second, add hosts and host name mapping (/etc/hosts), disable the firewall, disable SElinux, set multiple hosts to be secret free, and enable HTTP service.

192.168.242.134 hadoop1
Copy the code

Copy the code

Disable firewalldsystemctl stopFirewalldsystemctl stopfirewalldsystemctl stopFirewalldsystemctl stop Centos7 later systemctl disable firewalld. Service# effect restart # vi/etc/selinux/configSELINUX = disabledSELINUXTYPE = targetedCopy the code

The host is secret free:

For example, run the ssh-keygen -t rsa command on each host and press Enter three times to generate A hidden. SSH directory under the /home/csap directory. You need to grant permission to the directory 700: chmod 700. SSH. There are two files in this directory: id_RSA private key and id_rsa.pub public key. Sshtouch authorized_keyschmod 644 authorized_keys 3. For example, if you do not know the passwords of other machines, For example, bind 4A; Paste your id_Rsa. pub public key and the public keys of the other two machines into the authorized_keys file, and then copy this file to the other two machines. If all the hosts have public keys of other hosts, SSH secret-free login is complete. Ssh-copy-id -i ~/. SSH /id_rsa.pub? Ssh-copy-id -i ~/. [email protected]Copy the code

Clock synchronization (to ensure that other nodes clock the same as the ambari master node) :

# yum -y install ntp2 # yum -y install ntp2 # yum -y install ntp2 Server 127.127.1.0 # local clockfudge 127.127.1.0 stratum 103, synchronize the time of the primary node with that of other machines, vim /etc/ntp.conf, add the following: Xxx. xx4. Restart the NTP service systemctl restart NTPD or service NTPD restartSystemctl status NTPD or service NTPD Status5. Verify that NTPQ -p is executed to synchronize all nodes. An asterisk (*) on the left indicates that the synchronization is successful. If not successful; /usr/sbin/ntpDate stdtime.gov.hkntpDate XXX.xxx.xxx. XXX Manually synchronize timeCopy the code

If the HTTP service is enabled, cm and CDH directories are automatically created in /var/www/html/

Yum -y install httpdSystemctl start HTTPD or service HTTPD startCopy the code

Install JDK and configure JAVA_HOME.

# Uninstall other JDK, Use the following command to view all current Java environment RPM installation package - qa | grep java# using the following command to delete all the Java installation package RPM -e -- nodeps installation package name # jdkmkdir/data/JDKCD/data/jdktar -xvf jdk-8u144-linux-x64.tar.gz# modify the environment variable, configure the Java path vi ~/.bash_profile# add configuration: JAVA_HOME=/data/ JDK/jdk1.8.0_144path =$PATH:$HOME/bin:${JAVA_HOME}/bin:${JAVA_HOME}/sbin# enable source ~/. Bash_profile # Java -version Indicates whether the command takes effectCopy the code

3. Configure mariaDB. As hive and Ambari metadata

Install databaseCopy the code

4. Upload the installation package, create the local YUM source, and install ambari.

Configure the yum source (on all nodes), delete all repo files under /etc/yum.repos.d/, delete all repo files under /etc/yum.repos.d/, delete all repo files under /etc/yum.

# Mount file (note that this is used in effect, shutdown will be invalid. D /my.repo[myrepo]baseurl=file:///mntenabled=1gpgcheck=0Copy the code

HDP and downloaded to a local ambari RPM files uploaded to the/var/WWWW/HTML/ambari directory, a file is too large (> 4 g), xshell rz instructions cannot upload directly, you can refer to post: blog.csdn.net/FLawiet/art…

CD /var/www/ HTML /ambari/tar -zxvf ambari-2.6.1.5-centos7.tar.gz -c /var/www/ HTML/ambaritar-zxvf HDP - 2.6.4.0 - centos7 - RPM. Tar. Gz - C/var/WWW/HTML/ambarimkdir/var/WWW/HTML/ambari HDP/UTILS/tar ZXVF. - /var/ WWW/HTML /ambari/ hdp-utils / CD/var/WWW/HTML/ambari/find -name *. Html# HTML files on a folder; CD./HDP/centos7/2.6.4.0-91/rm *.html Copy the file to /etc/yum.repos.d/ as the local yum source. Repos. D: /etc/yum/repos. D: /etc/yum/repos. #VERSION_NUMBER=2.6.1.5-3[ambari-2.6.1.5]name=ambari Version - Ambari - 2.6.1.5 baseurl = http://192.168.242.111/ambari/ambari/centos7/2.6.1.5-3/gpgcheck=1gpgkey=http://192.168.242.111/amb Ari/ambari/centos7/2.6.1.5-3 / RPM - GPG KEY/RPM -- GPG KEY - Jenkinsenabled = 1 priority = 1 # (2) in the/etc/yum repos. D folder to revise the HDP. ', Ensure HTTP access to baseurl; Cat hdp.repo # contents are as follows #VERSION_NUMBER=2.6.4.0-91[HDP-2.6.4.0]name=HDP Version - HDP - 2.6.4.0 baseurl = http://192.168.242.111/ambari/HDP/centos7/2.6.4.0-91/gpgcheck=1gpgkey=http://192.168.242.111/ambari/H DP/centos7/2.6.4.0-91 / RPM - GPG KEY/RPM -- GPG KEY - Jenkinsenabled = 1 priority = 1 [HDP UTILS - 1.1.0.22] name = HDP - UTILS Version - HDP UTILS - 1.1.0.22 baseurl = http://192.168.242.111/ambari/HDP-UTILS/centos7/1.1.0.22# Baseurl = http://192.168.242.111/ambari/HDP-UTILSgpgcheck=1gpgkey=http://192.168.242.111/ambari/HDP-UTILS/centos7/1.1.0.22 /RPM-GPG-KEY/RPM-GPG-KEY-Jenkins# Gpgkey = http://192.168.242.111/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1Copy the code

The ambari-Server installation starts.

# yum install -y ambari-server# yum install -y ambari-serverCopy the code

The configuration prompts you for the login passwords of the JAVA_HOME and Ambari metadata databases:


   
    mysql -uroot -p123456use ambarisource /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sqlshow tables;
   
Copy the code

After the database is initialized, start ambari-server start. /var/log/ambari-server/ambari-server.log is a database permission problem, similar to CDH cm, web access after successful installation (admin, admin);

5. The AMbari interface is used to configure HDP.

Follow the instructions to select the location of the RMP package for HDP:

Possible error: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password) Error: netutil. py: 124-server athttps://hadoop1:8440is not reachable, sleeping for 10 seconds…” , None) Reference:Blog.csdn.net/qq\_4180551…

Component installation. This section uses HDFS, Mr, YARN, ZK, and Spark as an example to describe how to install components.

Good historical articles recommended

Building a computing and storage system for big data platform from zero to one
Build scheduling system of big data platform from 0 to 1
Build data acquisition system of big data platform from 0 to 1
How to build a big data platform from zero to one
Big data platform based on Cloudera Distribution Hadoop (CDH)

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Big data platform based on Ambari

1. Prepare the installation environment

1. Prepare the installation package

2. Prepare the host.

3. Configure mariaDB. As hive and Ambari metadata

4. Upload the installation package, create the local YUM source, and install ambari.

5. The AMbari interface is used to configure HDP.

Good historical articles recommended

Big data platform based on Ambari

1. Prepare the installation environment

1. Prepare the installation package

2. Prepare the host.

3. Configure mariaDB. As hive and Ambari metadata

4. Upload the installation package, create the local YUM source, and install ambari.

5. The AMbari interface is used to configure HDP.

Good historical articles recommended

Related Posts

Faced with the new technology of blockchain, traditional investors are anxious

Machine learning Note 3 — Neural networks

The sum of two Numbers