First, replica set

1.1 Replication Function

To ensure data security and high availability, MongoDB provides the replication function to replicate data on the primary node to multiple secondary nodes. In this way, even if the primary node is faulty, data is stored in multiple copies to ensure data security. A standard three-node replica set architecture is as follows:

The replication process is as follows:

1. Initial synchronization

At replica set initialization, the mongod process on the master node scans each collection in each database on the current node, and then sends this data to the slave node for a full initialized copy.

2. Synchronous replication

After initialization, the secondary node synchronously replicates oplog data of the primary node. Oplog (Operation log) is a special set that stores operation records of all data in the database. Each operation in Oplog is idempotent, meaning that each operation produces the same result whether applied once or more to the target data set.

3. Perform asynchronous execution

MongoDB groups each batch of operations by namespace or Document ID and applies each group of operations using different threads. Meanwhile, MongoDB applies write operations to documents on slave nodes strictly in the original write order.

1.2 arbiter

In addition to normal Mongod instances that can be added to the replica set, additional Mongod instances can be added to the replica set as arbiters. The arbiter does not write or write data, but only responds to election requests and participates in arbitration. Since it does not store data, it can take up fewer server resources, and if your replica set has an even number of members, you can ensure valid elections by adding arbiters.

Although the arbiter can take up fewer server resources, it does not contribute to data security because it does not store data. Therefore, you should avoid using arbitrators and try to use at most one arbitrator, adding an arbitrator if the number of nodes is even and not if the number of nodes is odd.

Fault discovery and recovery

2.1 Fault Discovery

Heartbeat detection is maintained between each node of the replica set every two seconds. When the communication time between the slave node and the master node exceeds the configured electionTimeoutMillis time (default is 10 seconds), the master node is considered to be faulty, and the replica set elects a new master node.

2.2 Preferential election

MongoDB’s election algorithm tries to give priority to high-priority nodes to initiate elections, thus making it easier to win elections. If a node with a lower priority is elected as the new primary for a short period of time, the replica set continues to initiate elections until the highest priority node available becomes the new primary. It is important to note that during this process, a member with priority 0 cannot seek election or become the primary node.

2.3 Voting Members

After a node initiates an election, nodes with voting rights are required to vote. When more than half of the votes are obtained, the standby node can become the new primary node. For a replicated set, only nodes in the following states have voting rights, which are collectively referred to as voting members:

  • PRIMARY: the PRIMARY node of the replica set.
  • SECONDARY: indicates the SECONDARY node in the replication state.
  • STARTUP2: After mongod completes configuration loading, each member of the replica set enters STARTUP2 state, at which point it becomes an active member of the replica set and is eligible to vote. The member then decides whether to perform the initial synchronization. If the member starts initial synchronization, the member will remain in STARTUP2 state until all data is replicated and indexes are built. And then the member transitions to RECOVERING.
  • RECOVERING: A replica set enters a RECOVERING state when its members are not yet ready to be read. Members in the recovered state are eligible to vote in the election, but are not eligible to become the master node.
  • ARBITER: The members of the ARBITER state do not copy data or accept writes. The ARBITER is usually in this state.
  • ROLLBACK: If a member is rolling back data, it is in the ROLLBACK state.

In addition to voting members, members that hold copies of the replica set’s data and can be read from client applications but do not vote are collectively referred to as non-voting members. Due to the cost of network communication, the replica set of MongoDB has a maximum of 50 nodes (members) and a maximum of 7 voting members by default. The requirements for voting and non-voting members are as follows:

  • Priority of non-voting membersmembers[n].priorityThe value must be 0. A member whose priority is 0 cannot seek election or become the primary node.
  • The number of votes held by a node whose priority is greater than 0members[n].votesThe value cannot be 0. The default value is 1. The optional value can be 1 or 0.

The following example is a nine-member replica set with seven voting members and two non-voting members:

3. Set up replica sets

Here, take building a three-node replica set as an example, using three servers, the host names are respectively hadoOP001, HadoOP002, hadoOP003.

3.1 Download and Decompress the file

Select the MongoDB version and download it from www.mongodb.com/download-ce…

Here, I have downloaded the 4.0.10 version and installed RHEL 7.0. Decompress the downloaded version:

Tar -zxvf mongodb- linux-x86_64-rhel70-4.0.0.tgz -c /usr/appCopy the code

3.2 Configuring Environment Variables

Configure environment variables:

vi /etc/profile
Copy the code
Export MONGODB_HOME = / usr/app/mongo - Linux - x86_64 - rhel70 4.0.10 / export PATH = ${MONGODB_HOME} / bin: $PATHCopy the code

Make configured environment variables take effect immediately:

source /etc/profile
Copy the code

3.3 Modifying The Configuration

The default MongoDB directory for storing data is /var/lib/mongo, and the default MongoDB directory for storing logs is /var/log/mongodb. When the TGZ installation package is used, the two directories are not automatically created. You need to manually create them. Since only temporary files can be stored under /var/, we use another directory to store them. The command is as follows:

mkdir -p /home/mongodb/data
mkdir -p /home/mongodb/log
Copy the code

If the TGZ installation package is used, the program does not automatically create a configuration file. You need to manually create a configuration file:

vim /etc/mongod.conf
Copy the code

Add the following configuration in the configuration file, which is in YAML format:

ProcessManagement: # later start fork the process way: true systemLog: destination: the file path: "/ home/mongo/log/mongod log" logAppend: True storage: dbPath: "/home/mongodb/data" net: port: 27017 All nodes in the same replica set need to ensure that the replica set name is the sameCopy the code

You can refer to the official MongoDB document Configuration File Options for all Configuration items

3.4 Starting the Service

The preceding configuration steps are the same on the three hosts. Then start the Mongod service on all three hosts with the following command:

mongod -f /etc/mongod.conf
Copy the code

3.5 Configuring a Replica Set

Connect to the local service using Mongo shell on any host, then configure the replica set directly using the following command:

rs.initiate( {
   _id : "rs0",
   members: [
      { _id: 0, host: "hadoop001:27017" },
      { _id: 1, host: "hadoop002:27017" },
      { _id: 2, host: "hadoop003:27017" }
   ]
})
Copy the code

3.6 Viewing the Replica Set

Use the rs.status() command to view the replica set status. It can be seen from the output that Hadoop001 is the PRIMARY node, while hadoop002 and HadoOP003 are both SECONDARY nodes, indicating that the replica set has been successfully built.

The resources

  • The official document: docs.mongodb.com/manual/repl…
  • The official configuration instructions: docs.mongodb.com/manual/refe…

For more articles, please visit the full stack Engineer manual at GitHub.Github.com/heibaiying/…