[toc]
This section describes IP address types in an Oracle RAC cluster
What are the functions of Public IP, Private IP, Virtual IP, SCAN IP, GNS VIP, and HAIP in Oracle RAC?
Starting from Oracle 11G, the RAC installation requires at least seven IP addresses and two nics (one public NETWORK NIC and one private network card). The public, VIP, and Scan nics are in the same network segment and use the public NETWORK NIC. The private nics are in another network segment and use the private network card. The host name cannot contain a dash, for example, RAC_01 is not allowed. Run the ifconfig -a command to check whether the network device names of the two nodes are the same. In addition, after the /etc/hosts file is configured and before RAC installation, four IP addresses of the public and private networks can be pinged. The remaining three IP addresses cannot be pinged.
Starting from Oracle 18C, you are advised to configure at least three scan nodes.
The FOLLOWING table shows the IP address planning during RAC installation:
The configuration in the /etc/hosts file is as follows:
[root@raclhr- 19c-n1 ~]# more /etc/hosts
127.0. 01. localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#Public IP
192.16859.52. raclhr- 19c-n1
192.16859.53. raclhr- 19c-n2
#Private IP
192.1682.52. raclhr- 19c-n1-priv
192.1682.53. raclhr- 19c-n2-priv
#Virtual IP
192.16859.54. raclhr- 19c-n1-vip
192.16859.55. raclhr- 19c-n2-vip
#Scan IP
192.16859.. 56 raclhr- 19c-scan
192.16859.57. raclhr- 19c-scan
192.16859.. 58 raclhr- 19c-scan
Copy the codeThe following describes several TYPES of IP in a RAC environment and their functions.
(1) Public IP
The Public IP address is the real IP address of the network adapter. Public IP addresses must be configured for each node before Oracle software is installed. Oracle provides network services through Public IP addresses. If the network adapter where the Public IP address resides in the RAC is faulty, the node cannot provide external services. Therefore, you are advised to enable multiple network adapter binding on the operating system layer to implement IP Failover. In a two-node RAC environment, the tnsnames.ora file contains the Public IP addresses, port numbers, and communication protocols of the two nodes. If load balancing is not enabled (software/hardware), then when the host corresponding to the first ADDRESS in tnsnames.ora fails or shuts down, Oracle waits for a network timeout while the client connects and continues to connect to the database instance corresponding to the second ADDRSS. Even if an RAC node is still available, users need to wait several seconds for each connection. Network timeouts can make the user experience quite bad. Therefore, the appearance of VIP (Virtual IP) in Oracle 10G RAC solves this problem perfectly.
Note that the Public IP address is not the Elastic Public IP address (EIP) on the cloud platform. On the cloud platform, an EIP provides independent Public IP address resources, including Public IP addresses and Public egress bandwidth services. You can flexibly bind and unbind resources such as elastic cloud servers, bare-metal servers, virtual IP addresses, elastic load balancers, and NAT gateways. An elastic public IP address can be bound to only one cloud resource.
You can use an EIP to connect to public network resources. This IP address is open to the public and is charged.
(2) Private IP
For Oracle clusters, private network communication is very important because most of the communication between nodes is carried out over private networks. Private network communication can be basically divided into two types: the first is the communication between the cluster level; The second is communication between database instances. The first communication (for example, network heartbeat between nodes) is characterized by continuous existence and high real-time requirements, but the amount of data is relatively small, so it can be transmitted through TCP/IP protocol. The second communication is the data transmission between instances caused by Cache Fusion, which is characterized by a large amount of data and high speed requirements. TCP/IP protocol can no longer meet the requirements, so UDP or RDS is used. Oracle also advises users to configure high availability and load balancing for private networks in clusters.
Like a Public IP address, a Private IP address is a Private network IP address or heartbeat IP address. It is also the real IP address of a network adapter. You must configure a Private IP address before installing the Oracle cluster software on each node. The Private IP address is used for tasks such as inter-cluster heartbeat synchronization and Cache Fusion. In Oracle 12C, the Private IP address is used for Flex ASM tasks. Of course, if you do not set a Private IP but a Public IP can also do these tasks. However, the network adapter where the Public IP address resides will be overloaded. If the network adapter fails, the cluster will split and remove some nodes to ensure consistency, resulting in unstable RAC performance. Starting from Oracle 11G, you need to configure a Private IP address.
For Oracle 10G and 11gR1 clusters, Oracle does not provide the high availability and load balancing features of private networks. Instead, users are advised to configure these features at the operating system level (such as Linux bonding and AIX Etherchannel). In this way, the binding technology of multiple network adapters at the operating system level is enabled to implement IP Failover. The HAIP technology, launched from Oracle 11.2.0.2, provides high availability and load balancing features on private networks. It replaces network adapter binding technology at the operating system level and is more powerful and compatible. However, HAIP is prone to bugs and will be disabled when you install RAC on cloud platforms.
(3) Virtual IP (VIP)
VIP is a new feature in Oracle 10G RAC called virtual IP. VIP is a virtual IP address created by the Oracle cluster software on the network adapter where the Public IP address resides. The IP address must be in the same subnet as the Public IP address. Before installing Oracle cluster software, you only need to define the /etc/hosts file. Under normal circumstances, VIP and Public IP functions exactly the same. The background process PMON registers instance information with the listener where the VIP of each node resides. The local listener displays two addresses host, one Public IP address and the other VIP address. When a node fails, the Oracle cluster software automatically floats the VIP to other nodes, but the local listener does not float to other nodes. In a two-node RAC architecture, if the first node fails, the second node has two VIPs. If the client fails to connect to the first VIP, the client immediately connects to the corresponding instance of the second VIP. The whole switching process is very short and the user will not feel any node failure in the RAC architecture. The whole connection process is transparent to the user. However, it is not transparent in the true sense. Users can still know how many nodes the entire RAC architecture is composed of. Therefore, SCAN IP is a new concept introduced in Oracle 11G RAC to achieve transparency of user connections.
VIP features:
(1) VIP is created by VIPCA script.
(2) VIP registers with OCR as A CRS Resource of Nodeapps type, and CRS maintains the status.
(3) VIP is bound to the Public nic of the node, so the Public NIC has two addresses.
(4) When a node fails, CRS will transfer the VIP of the failed node to other nodes.
(5) The LISTENER on each node listens for both the Public IP address and VIP on the Public nic.
(6) Tnsnames. ora on the client is usually configured with a VIP pointing to a node.
You can use the following command to forcibly migrate the VIP from node 1 to node 2:
crsctl relocate resource ora.raclhr- 11gr2-n1.vip -n raclhr- 11gR2-N2 -f
Copy the codeThe following command is for Oracle 11g prior to release:
crs_relocate ora.lhr- 11gr2-rac1.vip -c lhr- 11gR2-rac2 -f
Copy the code(4) SCAN IP
The SCAN (Single Client Access Name) IP address was introduced from Oracle 11gR2 RAC, which is equivalent to adding a virtual network service layer between the Client and the database. SCAN IP and SCAP IP LISTENER. In the tnsnames.ora configuration file on the client, you only need to configure SCAN IP. Then users can access the database and load balancing is implemented. Clients use SCAN IP and SCAN IP LISTENER to load balance connections to the RAC database. The advantage of using SCAN IP compared to previous versions of RAC is that client configuration information does not need to be modified when nodes are added or deleted from the background RAC database. SCAN provides a domain name to access RAC. A domain name can resolve one to three (note, a maximum of three) SCAN IP addresses, which can be resolved using DNS, GNS, or /etc/hosts files. Note that SCAN IP, VIP, and Public IP must belong to the same subnet.
SCAN listeners can listen to all databases running in the cluster, which is the principle of SCAN load balancing. SCAN listeners are randomly assigned to node servers along with SCAN VIP. If a node fails, the SCAN VIP running on this node will drift. In this case, SCAN listeners also drift to normal nodes and continue to listen for connection requests for SCAN VIP. The next time the PMON process dynamically updates the instance information to the SCAN listener, it receives the client connection again. This is different from VIP drift.
To connect to the RAC database using SCAN IP and implement load balancing, configure the remote_listener parameter in the database:
alter system set remote_listener='raclhr-19c-scan:1521' sid=The '*';
Copy the codeRaclhr-19c-scan is the scan name of the cluster.
By setting this parameter, any database instance is registered with the SCAN listener, which is mainly used to register remote database instances, so the SCAN listener can load balance the distribution of connection requests to the node local listener, that is, to the instance on its local node.
The listening status of the SCAN listener is as follows:
[root@raclhr- 19c-n1 ~]# lsnrctl status LISTENER_SCAN1
LSNRCTL for Linux: Version 19.0. 0. 0. 0 - Production on 02-MAR- 2021. 15:06:21
Copyright (c) 1991.2019, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias LISTENER_SCAN1
Version TNSLSNR for Linux: Version 19.0. 0. 0. 0 - Production
Start Date 02-MAR- 2021. 09:54:36
Uptime 0 days 5 hr. 11 min. 45 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/19.3. 0/grid/network/admin/listener.ora
Listener Log File /u01/app/grid/diag/tnslsnr/raclhr- 19c-n1/listener_scan1/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.16859.. 56)(PORT=1521)))
Services Summary...
Service "86b637b62fdf7a65e053f706e80a27ca" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
Service "9188f464f9113646e053343ba8c0fe51" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
Service "pdblhr1" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
Service "rac19c" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
Service "rac19cXDB" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
Service "rac19c_CFG" has 2 instance(s).
Instance "rac19c1", status READY, has 1 handler(s) for this service...
Instance "rac19c2", status READY, has 1 handler(s) for this service...
The command completed successfully
Copy the codeNext, do the load balancing test:
[root@raclhr- 19c-n2 ~]# more lhr_load_banlance.sh
#!/bin/bash
for i in {1.100.}
do
echo $i
sqlplus -S "sys/[email protected] / rac19c as sysdba." "<<EOF
select instance_name from v\$instance;
EOF
done
exit 0
EOF
[root@raclhr- 19c-n2 ~]# chmod +x lhr_load_banlance.sh
[root@raclhr- 19c-n2 ~]# ./lhr_load_banlance.sh > a.txt
[root@raclhr- 19c-n2 ~]# more a.txt | grep rac19c1 | wc -l
42
[root@raclhr- 19c-n2 ~]# more a.txt | grep rac19c2 | wc -l
58
Copy the codeAs you can see, SCAN IP is basically load-balancing the connection to the two nodes in rac.
5. GNS VIP
GNS VIP is a new feature for Oracle 11G RAC. In the traditional RAC architecture, Public IP addresses, Private IP addresses, Virtual IP addresses, and SCAN IP addresses are pre-configured. If the GNS service is enabled, you only need to configure the Public IP address and Private IP address. The Virtual IP address and SCAN IP address are dynamically obtained by the GNS service from the DHCP server. Before configuring the GNS service, you need to configure the DNS and DHCP services. GNS VIP can be bound to the Public NIC of any node to implement the GNS service. If the GNS service is not configured, you do not need to configure GNS VIP. Configuration is a bit more complex than traditional architectures, so you rarely see GNS VIP in Oracle 11G RAC architectures. The 12C Flex Cluster architecture must be configured with GNS VIP.
(6) Highly Available IP
Before Oracle 11.2.0.2, private network redundancy was generally realized by network adapter binding (such as Bond) on the OS. Since Oracle 11.2.0.2, HAIP (Highly Available IP) technology has been introduced to replace network adapter binding technology at the operating system level. More powerful, more compatible. HAIP implements high availability and load balancing of inter-cluster links by providing unique IP addresses in the 169.254.* network segment. Therefore, when installing RAC 11.2.0.2 or later, ensure that IP address 169.254.* is not occupied. HAIP technology can no longer rely on any third-party redundancy technology to achieve the redundancy of private network cards.
The resource ora.cluster_interconnect.haip will start with one to four local HAIP addresses attached to the Private network adapter (Private network card). Complete internal communication of Oracle RAC and ASM through HAIP. If a private network card is physically damaged, the HAIP address on the card will migrate to another available private network. Multiple private network cards can be defined during installation or after GRID configuration by calling the $GRID_HOME/bin/oifcfg setif tool (command: Oifcfg setif -global eth2/192.168.1.0:cluster_interconnect) to configure HAIP.
The number of HAIP depends on the number of private network cards activated by the GRID. If there is only one private network card, the GRID will create one HAIP. If there are two private network cards, the GRID will create two HAIP’s. If there are more than two private network cards, the GRID creates four HAIP. The GRID supports up to four private network cards, and the actual number of HAIP addresses used by the cluster depends on the number of private network cards active on the first node in the cluster to start. If more private network cards are selected as Oracle’s private network, the more than four cards cannot be activated.
Ora. cluster_interconnect.haip is managed by the ohasd.bin process. The corresponding log in $GRID_HOME/log / $HOSTNAME/ohasd ohasd. Log and $GRID_HOME/log / $HOSTNAME/agent/ohasd/orarootagent_root/orarootagent_r Oot. Log in both locations. $GRID_BASE/diag/$HOSTNAME/ CRS /trace
[root@raclhr- 19c-n1 ~]# adrci
ADRCI: Release 19.0. 0. 0. 0 - Production on Tue Mar 2 15:31:08 2021
Copyright (c) 1982.2019, Oracle and/or its affiliates. All rights reserved.
ADR base = "/u01/app/grid"
adrci> show home
ADR Homes:
diag/asmcmd/user_oracle/raclhr- 19c-n1
diag/asmcmd/user_grid/raclhr- 19c-n1
diag/kfod/raclhr- 19c-n1/kfod
diag/tnslsnr/raclhr- 19c-n1/listener
diag/tnslsnr/raclhr- 19c-n1/listener_scan2
diag/tnslsnr/raclhr- 19c-n1/asmnet1lsnr_asm
diag/tnslsnr/raclhr- 19c-n1/listener_scan3
diag/tnslsnr/raclhr- 19c-n1/listener_scan1
diag/asm/+asm/+ASM1
diag/rdbms/_mgmtdb/-MGMTDB
diag/clients/user_root/host_4079108462_110
diag/clients/user_grid/host_4079108462_110
diag/crs/raclhr- 19c-n1/crs
[root@raclhr- 19c-n1 trace]# pwd
/u01/app/grid/diag/crs/raclhr- 19c-n1/crs/trace
[root@raclhr- 19c-n1 trace]# ll *ohasd*
-rw-rw---- 1 root oinstall 1358974 Mar 2 15:32 ohasd_cssdagent_root.trc
-rw-rw---- 1 root oinstall 262948 Mar 2 15:32 ohasd_cssdagent_root.trm
-rw-rw---- 1 root oinstall 1323737 Mar 2 15:32 ohasd_cssdmonitor_root.trc
-rw-rw---- 1 root oinstall 256254 Mar 2 15:32 ohasd_cssdmonitor_root.trm
-rw-rw---- 1 grid oinstall 2213526 Mar 2 15:32 ohasd_oraagent_grid.trc
-rw-rw---- 1 grid oinstall 405923 Mar 2 15:32 ohasd_oraagent_grid.trm
-rw-rw---- 1 root oinstall 17569942 Mar 2 15:32 ohasd_orarootagent_root.trc
-rw-rw---- 1 root oinstall 2998382 Mar 2 15:32 ohasd_orarootagent_root.trm
-rw-rw---- 1 root oinstall 7550545 Mar 2 15:32 ohasd.trc
-rw-rw---- 1 root oinstall 1170542 Mar 2 15:32 ohasd.trm
Copy the codeAfter the HAIP resource is online, you can run the ifconfig -a command to view many virtual nics similar to eth1:1. The HAIP address is 169.254.x. x. You can also view the HAIP address in the GV$CLUSTER_INTERCONNECTS view at the database level. The HAIP address is automatically assigned by the system and cannot be manually specified.
SQL> select * from GV$CLUSTER_INTERCONNECTS;
INST_ID NAME IP_ADDRESS IS_ SOURCE CON_ID
---------- --------------- ---------------------------------------------- --- ------------------------------- ----------
2 ens37:1 169.25412.150. NO 0
1 ens37:1 169.25415.64. NO 0
Copy the codeOracle databases and ASM instances can use HAIP to implement high availability and load balancing for private network communications. The traffic of the private network is load balanced on these private network cards. If a network card fails, the HAIP on it will automatically switch to another available private network card, thus not affecting the communication of the private network. Windows does not currently support HAIP technology.
However, in some customer environments, private networks are allocated through vlans. For network management requirements, the VLAN IP address must be bound to the network adapter, and the private IP address must be fixed (although the private network should be isolated according to the installation requirements of Oracle RAC). In this case, HAIP cannot be assigned. ASM resources that depend on it cannot be started. HAIP has a number of bugs, so you can disable HAIP if you are unlucky enough to encounter them. Teacher mai had already met 2 times at present, and check reason check for a long time, ashamed ah.
If the user uses an operating system-level binding or does not use a private network binding, the HAIP can be overridden by setting cluster_interconnect to specify a private network address in the RDBMS and ASM parameter files (if there are multiple private network addresses, separate them with colons). HAIP itself still exists, but ASM instances and RDBMS instances will not use HAIP in the future.
-- Modify DB parameters
alter system set cluster_interconnects="192.168.2.52" scope=spfile sid='rac19c1';
alter system set cluster_interconnects="192.168.2.53" scope=spfile sid='rac19c2';
-- Modify ASM parameters
alter system set cluster_interconnects="192.168.2.52" scope=spfile sid='+ASM1';
alter system set cluster_interconnects="192.168.2.53" scope=spfile sid='+ASM2';
-- Restart the cluster
crsctl stop has -f
crsctl start has
-- DB and ASM queries
SQL> select * from GV$CLUSTER_INTERCONNECTS;
INST_ID NAME IP_ADDRESS IS_ SOURCE CON_ID
---------- --------------- ---------------------------------------------- --- ------------------------------- ----------
2 ens37 192.1682.53. NO cluster_interconnects parameter 0
1 ens37 192.1682.52. NO cluster_interconnects parameter 0
Copy the codeDisable HAIP
In addition to changing the Cluster_Interconnect parameter above, we can also choose to disable HAIP. After HAIP is disabled, the ASM and DB use the private network NIC address by default. To disable HAIP, perform the following steps.
Run the following command as user root:
1. Stop CRS on all nodes
crsctl stop crs
Copy the code2. Run the following commands on each node in sequence (run the following commands on node 1 and then on node 2) :
crsctl start crs -excl -nocrs
crsctl stop res ora.asm -init
crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=0" -init
crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(i ntermediate:ora.cssd)'"-init
crsctl stop crs
Copy the codeOra. cluster_interconnect.haip has the original ENABLED value of 1:
ENABLED=1
Copy the codeThe original values of START_DEPENDENCIES and STOP_DEPENDENCIES for the ORa. asm resource are:
START_DEPENDENCIES=hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.cluster_interconnect.haip,ora.drivers.acfs)
STOP_DEPENDENCIES=hard(intermediate:ora.cssd,shutdown:ora.cluster_interconnect.haip)
Copy the code3. Start CRS on each node in turn
crsctl start crs
Copy the code4. Check whether HAIP is disabled
crsctl stat res -t -init
Copy the codeIf ora.cluster_interconnect.haip is offline, it is disabled.
ora.cluster_interconnect.haip
1 ONLINE OFFLINE STABLE
Copy the codeThen run the following command:
The ifconfig - a or IP - a | grep 169.254Copy the codeCheck if there is an address starting with 169.254. If there is no address, it indicates that the address has been disabled successfully.
The command to view the property values of the resource is given below:
[root@rac2 ~]# crsctl stat res ora.cluster_interconnect.haip -p -init | grep ENABLED
ENABLED=0
[root@rac2 ~]# crsctl stat res ora.asm -p -init | grep START_DEPENDENCIES
START_DEPENDENCIES=hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.drivers.acfs)
[root@rac2 ~]#
[root@rac2 ~]# crsctl stat res ora.asm -p -init | grep STOP_DEPENDENCIES
STOP_DEPENDENCIES=hard(intermediate:ora.cssd)
Copy the codeIf HAIP is enabled, the procedure is the same as that of disabled. The main modification commands are as follows:
crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=1" -init crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.cluster_interconnect.haip,ora.drivers.ac fs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd,shutdown:ora.cluster_interconnect.haip)'" -initCopy the codeDescription:
For more information about IP in RAC, see:
Blog.itpub.net/26736162/vi… Blog.itpub.net/26736162/vi… Blog.itpub.net/26736162/vi… Blog.itpub.net/26736162/vi… Blog.itpub.net/26736162/vi…
About Me
● The author of this article: Wheat Seedling, part of the content is arranged from the network, if there is infringement, please contact wheat seedling to delete ● This article in personal wechat public number (DB treasure) has been updated simultaneously ● QQ group number: 230161599, 618766405, wechat group private chat ● Personal QQ number (646634621), wechat account (db_bao), indicate the reason for adding ● Completed in Xi ‘an in March 2021 ● Updated date: March 2021 ● All rights reserved
● Wheat Seedling wechat store: weidian.com/?userid=793… Low wheat seedlings in the publication of the database class series: blog.itpub.net/26736162/vi… Low wheat seedlings OCP, OCM, high availability, the DBA course (Oracle, MySQL and no () : blog.itpub.net/26736162/vi… Low database written interview question and answer: mp.weixin.qq.com/s/Vm5PqNcDc…
Use wechat client to scan the following TWO-DIMENSIONAL code to follow the wechat public account of Wheat Seedling (DBbao) and QQ group (DBA Baodian), add Wheat seedling wechat, learn the most practical database technology.