Ali Cloud MongoDB use

  • Ali Cloud can be purchased directly by MongoDB’s service, and can be used directly after the purchase. The advantage of buying a service is that you don’t have to build it yourself.
  • Mongodb&java & NEtty&RPC High performance learning Communication Group: 876280300
  • High-performance Mongodb ORM framework zfoo open source address

I. Establishment of replica set

1. Prepare the MongoDB environment

  • Set the static IP addresses of the three VMS to
192.168.1.30 mongodb0
192.168.1.31 mongodb1
192.168.1.32 mongodb2
Copy the code
  • After setting the IP address, you need to match the IP address with the host name
Mongodb0 =mongodb0, mongodb0 =mongodb0, /etc/sysconfig/network Mongodb0 192.168.1.31 -> HOSTNAME=mongodb1 192.168.1.32 -> HOSTNAME= mongodbvim /etc/hosts, change to the host name we set, Mongodb0 localhost mongodb0 localhost4 localhost4. Localdomain4 ::1 localhost mongodb0 localhost6 Localhost6. Localdomain6 192.168.1.30 mongodb0 192.168.1.31 mongodb1 192.168.1.32 mongodb2Copy the code
  • It takes effect after the server is restarted. Run the hostname command to verify the hostname and the ping command to verify that the three VMS can ping each other using the hostname.
  • Enable firewall ports.

2. Create a directory and modify the configuration file

  • Mongodb0, mongodb1, mongodb2 use the command
Mkdir -p /data/mongodb/db mkdir -p /data/mongodb/logsCopy the code
  • Vim/usr/local/mongo/mongo. Config, modify the configuration file is as follows
dbpath=/data/mongodb/db logpath=/data/mongodb/logs/mongodb.log logappend=true port=22400 fork=true # # keyFile opens by default auth = true auth # keyFile = = true/data/mongo/key/myKey. TXT replSet = myReplicaSet # production environment need to set up IP: Bind_ip =127.0.0.1, the local IP address bind_IP =0.0.0.0Copy the code

3. Initialize the replica set

  • Clear the historical data of the directory, because it may be used by a single node in the past, cluster deployment may have unexpected problems, so clear the data first
/data/mongodb/db
/data/mongodb/logs
Copy the code
  • Mongodb0, mongodb1, mongoDB2, respectively
systemctl restart mongodb
Copy the code
  • Creating a replica set
Mongo --port mongodb0 # mongodb0 config = {_id: "myReplicaSet", members: [{_id: 0, host:"mongodb0:22400"}, {_id: mongodb0: mongodb0 1, host:"mongodb1:22400"}, {_id: 2, host:"mongodb1:22400", arbiterOnly: true}]} rs. Initiate (configCopy the code
  • Observing replica sets
Rs.conf () # replica set configuration rs.status() # replica set statusCopy the code

4. After the replica set is initialized, create accounts and passwords for the entire replica set

  • Create Key, mongodb0, mongodb1, mongodb2 using commands
echo "replicaSet key" > /data/mongodb/key/myKey.txt
chmod 600 /data/mongodb/key/myKey.txt
Copy the code

  • On the primary node, connect to the client and create user permissions (rs.status() for the primary node). See mongodb-setup.md for creating a user

  • Mongodb0, Mongodb1, and mongoDB2 are respectively closed for the replica set

systemctl stop mongodb
Copy the code
  • Vim/usr/local/mongo/mongo. Config, modify the configuration file is as follows
dbpath=/data/mongodb/db logpath=/data/mongodb/logs/mongodb.log logappend=true port=22400 fork=true # keyFile opens by default auth = true auth = true keyFile = / data/mongo/key/myKey. TXT replSet = myReplicaSet # production environment need to set up IP: Bind_ip =127.0.0.1, the local IP address bind_IP =0.0.0.0Copy the code
  • Mongodb0, Mongodb1, mongoDB2, respectively
systemctl restart mongodb
Copy the code

5. Data synchronization test

  • First write a piece of data to the PRIMARY(PRIMARY node)
use test
db.say.insert({"text":"Hello World"})
db.say.find()
Copy the code
  • Enter the SECONDARY node to check whether data is synchronized
By default, the SECONDARY node cannot read or write data. To read data from the SECONDARY node, slaveOk must be set to true. ReplSet can have only one Primary node, and data can only be written to the Primary node, not the SECONDARY node. SetSlaveOk to true db.getmongo ().setslaveok (); use test; db.say.find();Copy the code

6. Failover test

  • Turn off the primary node to see if the sub-set point can take over the work of the primary node

Two, principle analysis

1. Replica set realizes data synchronization, failover, read and write separation of database

The replica set should contain at least three nodes, one of which must be the master node.

Data synchronization: The slave node synchronizes oplog from the master node to its own node. Oplog has its own size, which can be set by –oplogSize. Reads and writes between master and slave are transparent to the programmer. Failover (DISASTER recovery) : A heartbeat communication occurs between the primary and secondary nodes within two seconds. If the primary node is found to be down, the primary node is selected from the node with the latest data and the highest weight. If the slave node fails, the slave node is directly selected and the data of the master node is synchronized to the slave node. Read/write separation: You can set whether to read data from the primary node or secondary node preferentially. Note: In a single node, data is first written to the memory and operations are stored in the memory view. The memory view is written to the journal log of the disk within 100ms. Mongodb synchronizes data in the memory to the disk every 60 seconds.

3. Management of replica sets

1. The diagnosis

  • View the replication status of the primary database
db.printReplicationInfo() configured oplog size: 990MB log length start to end: 174676SECS (48.52 HRS) Oplog First event time: Wed May 01 2019 13:29:38 GMT+0800 Oplog last event time: Fri May 03 2019 14:00:54 GMT+0800 now: Fri May 03 2019 14:00:56 GMT+0800 oplog size 990MB log length start to end = oplog last event time - oplog first event Time Because oplog is a fixed size, there is a loop, so must wait for oplog cycle several times, oplog and the actual match. If oplog is too small or data is written too fast, the logs in Oplog may be overwritten. When the slave node attempts to synchronize, it finds that the slave node cannot find the oplog and cannot keep up with the master node, so synchronization stops. If the log length start to end is very close to the slave node behind the primary, oplog may be too smallCopy the code
  • View the status of replication from the database
db.printSlaveReplicationInfo()

source: mongodb-config1:22400
	syncedTo: Fri May 03 2019 13:56:31 GMT+0800
	0 secs (0 hrs) behind the primary 
source: mongodb-config2:22400
	syncedTo: Fri May 03 2019 13:56:31 GMT+0800
	0 secs (0 hrs) behind the primary 

Copy the code

2. The replica set rolls back the lost data

The write recovery operation is rolled back when the primary node fails and is re-added to the replica set. A rollback is required only if the master node accepts a write request and the master node fails before the slave node replicates. When the master node rejoins the replica set as a slave node, it restores or rolls back its write operations to maintain database consistency with other members. When a rollback does occur, MongoDB writes the rollback data to the BSON file in the rollback folder of the database's dbPath, which can be read and retrieved using bsondump. The rollback file name is in the following format: <database>.<collection>.<timestamp>.bsonCopy the code