This is the sixth day of my participation in Gwen Challenge

Set up the Mongo cluster

One master, one subordinate, one arbitration mode. Mongodump exports the Bson format, which is binary. This tool is useful for recovering entire instances, individual databases, and specified collections. They can be used to back up live running databases (without locking or shutting them down)

The installation

download

Wget HTTP: / / https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel80-4.2.8.tgzCopy the code

Unpack the

The tar - ZXF mongo - Linux - x86_64 - rhel80-4.2.8. TGZCopy the code

CD mongo - Linux - x86_64 - rhel80 4.2.8 /Copy the code

Configuring the Active Node

vim mongodb.conf
Copy the code

The conf content is as follows:

port=27018 dbpath=/usr/local/mongo/db/master logpath=/usr/local/mongo/logs/master/mongodb.log pidfilepath=/usr/local/mongo/pid/master/mdb.pid logappend=true fork=true maxConns=100 #noauth=false journal=true StorageEngine = wiredTiger bind_ip = 0.0.0.0 replSet = rs0Copy the code

Creating a Soft connection

ln -s /usr/local/mongo/master/bin/mongod /usr/bin/mongodmaster
Copy the code

ln -s /usr/local/mongo/master/bin/mongo /usr/bin/mongomaster
Copy the code

Configuring slave Nodes

cd /usr/local/mongo/
Copy the code

mkdir slave
Copy the code

Copy the master configuration file

cp -r master/* slave
Copy the code

Modify the

cd slave
vim mongodb.conf
Copy the code

As follows:

port=27019 dbpath=/usr/local/mongo/db/slave logpath=/usr/local/mongo/logs/slave/mongodb.log pidfilepath=/usr/local/mongo/pid/slave/mdb.pid logappend=true fork=true maxConns=100 #noauth=false journal=true StorageEngine = wiredTiger bind_ip = 0.0.0.0 replSet = rsss0Copy the code

Creating a Soft connection

ln -s /usr/local/mongo/slave/bin/mongod /usr/bin/mongodslave
ln -s /usr/local/mongo/slave/bin/mongo /usr/bin/mongoslave
Copy the code

Configuring quorum Nodes

cd /usr/local/mongo/
Copy the code

mkdir arbiter
Copy the code

Copy the master configuration file

cp -r master/* arbiter
Copy the code

Modify the

cd arbiter
vim mongodb.conf
Copy the code

As follows:

port=27017 dbpath=/usr/local/mongo/db/arbiter logpath=/usr/local/mongo/logs/arbiter/mongodb.log pidfilepath=/usr/local/mongo/pid/arbiter/mdb.pid logappend=true fork=true maxConns=100 #noauth=false journal=true StorageEngine = wiredTiger bind_ip = 0.0.0.0 replSet = rs0Copy the code

Creating a Soft connection

ln -s /usr/local/mongo/arbiter/bin/mongod /usr/bin/mongodslave
ln -s /usr/local/mongo/arbiter/bin/mongo /usr/bin/mongoslave
Copy the code

Start the service

master

cd /usr/local/mongo/master/
mongodmaster --config mongodb.conf
Copy the code

slave

cd /usr/local/mongo/slave/
mongodslave --config mongodb.conf
Copy the code

arbiter

cd /usr/local/mongo/arbiter/
mongodarbiter --config mongodb.conf
Copy the code

The configuration relationship

Mongomaster -port 27018 -host 127.0.0.1Copy the code

rs.initiate({ _id:"rsss0", members:[ {_id: 0, host: 127.0.0.1: "27018", priority: 2}, {_id: 1, host: 127.0.0.1: "27019", priority: 1}, {_id: 2, host: "127.0.0.1:27017", arbite rOnly:true} ] });Copy the code

Check the replication set status: rs.status(); To check whether the current node is the master node: rs.ismaster ();

Read from slave node: rs.slaveok (true)

Rs. Initiate ({_id: “rsss0 members: [{_id: 0, host: 127.0.0.1:” 27017 “, priority: 2}]});

The backup data

Write data

Mongomaster -port 27018 -host 127.0.0.1 use test db.test.insert({"id":3}) exitCopy the code

CD/usr/local/mongo/master/bin mongodump -h 127.0.0.1: test - 27018 - d o/usr/local/mongo/dbdumpCopy the code

Delete the original data file.

Mongomaster -port 27018 -host 127.0.0.1 db.test.remove({'id':1}) exitCopy the code

use

127.0.0.1 mongorestore - h - 27018 - d test/usr/local/mongo/dbdump/test/test. The bsonCopy the code

Data is successfully recovered. Procedure

Configuring Authentication

Mongomaster 127.0.0.1:27018 use admin db.createuser ({user: "user", PWD: "PWD ", roles: [{role: "root", db: "admin"} ] } ) exitCopy the code

cd /usr/local/mongo

mkdir keyfile

cd keyfile

openssl rand -base64 741 > mongo.key
ls# Change the read/write permissions of the file to too open.
# error:Permissions on the/data/mongomaster/mongo - Linux - x86_64 - rhel70-4.2.9 / bin/keyfile/mongo. Key are too openCopy the code

cd /usr/local/mongo/master

vim mongo.conf
Copy the code

keyFile=/usr/local/mongo/keyfile/mongo.key
clusterAuthMode=keyFile
auth=TRUE
Copy the code

Modify the configuration files of slave and arbiter in the same way

The final configuration file is as follows

port=27017
dbpath=/usr/local/mongo/db/arbiter
logpath=/usr/local/mongo/logs/arbiter/mongodb.log
pidfilepath=/usr/local/mongo/pid/arbiter/mdb.pid
keyFile=/usr/local/mongo/keyfile/mongo.key
logappend=true
fork=true
maxConns=100
#noauth=falseClusterAuthMode =keyFile Auth =TRUE Journal =TRUE storageEngine=wiredTiger BIND_IP =0.0.0.0 replSet= rSSS0Copy the code

Once you restart the service, you must authenticate.

If you do not have permission to log in, you can log in from admin, and then switch to the new library to create a new user

use admin db.auth("","") use test db.createUser( ... {... user: "user", ... pwd: "pwd", ... roles: [ { role: "readWrite", db: "ziot" } ] ... }...). # #Copy the code

Mongodump backup

Writing shell scripts

vim dump.sh

#! /bin/bash
#Mongodump command pathDUMP = / usr/local/mongo/mongo - Linux - x86_64 - rhel80-4.2.8 / bin/mongodump#Tar backup package Temporary backup directory. The tar package must be periodically transferred to local storage for backup
OUT_DIR=/usr/local/mongo/dump
#Full backup directory path
TAR_DIR=/usr/local/mongo/dumps
#Obtain the current system time
DATE=$(date +%Y%m%d%H%M)
#Database account
DB_USER=user
#Database password
DB_PASS=pwd
#DAYS=15 Indicates that backup generated 15 DAYS ago is deleted. That is, only backup generated in the last 15 DAYS is retained
DAYS=15
#The final saved database backup file
TAR_BAK="mongodb_bak_$DATE.tar.gz"
cd $OUT_DIR
# rm -rf $OUT_DIR/ *if [ ! -d testgrid ]; then mkdir -p $OUT_DIR/$DATE else echo dir exist fi#Backing up all databases
$The DUMP - h 192.168.1.18:27017 - u$DB_USER -p $DB_PASS -o $OUT_DIR/$DATE
#Gz format
tar -zcvf $TAR_DIR/$TAR_BAK $OUT_DIR/$DATE
#Delete backup files generated 15 days ago
#find $TAR_DIR/ -mtime +$DAYS - delete
#Delete the tar backup file created 10 days ago
#find $OUT_DIR/ -mtime + 10 -name "*.tar.gz" -exec rm -rf {} \;
exit
Copy the code

Run the crontab periodically

crontab -e

The full MongoDB backup script will be executed at 3am every day
0 3 * * *  sh  /data/script/mongodbfullbackup.sh    >/dev/null 2>&1
Copy the code

Mongodump common command

The parameters of the mongodump

parameter	Parameters that
-h	Specifies the IP address of the database host
-u	Specifies the user name of the database
-p	Specifies the password for the database
-d	Specifies the name of the database
-c	Specify the name of the collection
-o	Specifies the file name to export
-q	Specifies the filter criteria for the exported data
–authenticationDatabase	Validate the name of the data
–gzip	Backup time compression
–oplog	use oplog for taking a point-in-time snapshot

Mongodump **** parameter practice

Whole database backup

Mongodump -h IP address :27017 -uroot -proot --authenticationDatabase admin -o /home/mongod/backup/fullCopy the code

Backup the test library

Mongodump -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -o /home/mongod/backup/Copy the code

Backed up the vast collection under the test library

Mongodump -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -c vast -o /home/mongod/backup/Copy the code

Compressed backup library

Mongodump -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -o /home/mongod/backup/ --gzipCopy the code

Compress the backup table

Mongodump -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -c vast -o /home/mongod/backup/ --gzipCopy the code

Mongorestore Restore practice

Mongorestore and mongoimport are similar parameters

parameter	Parameters that
-h	Specifies the IP address of the database host
-u	Specifies the user name of the database
-p	Specifies the password for the database
-d	Specifies the name of the database
-c	Specify the name of the collection
-o	Specifies the file name to export
-q	Specifies the filter criteria for the exported data
–authenticationDatabase	Validate the name of the data
–gzip	Backup time compression
–oplog	use oplog for taking a point-in-time snapshot
–drop	Drop the previous collection when resuming

Restore single repository in full repository backup (based on previous full repository backup)

Mongorestore - h IP address: 27017 - uroot - proot, authenticationDatabase admin - d test - drop/home/mongod/backup/full/test /Copy the code

Recovery test library

Mongorestore - h IP address: 27017 - uroot - proot, authenticationDatabase admin - d test/home/mongod/backup/test /Copy the code

Restore the vast collection under the test library

Mongorestore -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -c vast /home/mongod/backup/test/vast.bsonCopy the code

— Drop Parameter practice recovery

# restore single library mongorestore - h IP address: 27017 - uroot - proot, authenticationDatabase admin - d test - drop/home/mongod/backup/test / Mongorestore -h IP address :27017 -uroot -proot --authenticationDatabase admin -d test -c vast --drop /home/mongod/backup/test/vast.bsonCopy the code

Comparison between Mongo backups

The original

Mongoexport /mongoimport imports/exports JSON format, while mongodump/mongorestore imports/exports BSON format.

JSON is readable but bulky, while BSON is binary, small but barely readable to humans.

Depending on compatibility between mongodb versions, the BSON format may vary from version to version, so mongodump/ Mongorestore may not be successful between versions. Using JSON formats, mongoexport/ Mongoimport, is an option when cross-version data migration is not possible using BSON. Mongodump/MongoRestore across versions is not recommended. If you do, check the documentation to see if the two versions are compatible (most of the time).

JSON has good cross-version commonality, but it only retains the data part, not indexes, accounts and other basic information. It should be used with care.

Python connects to Mongo as a replica set

def client(self, db) :
    # motor
    self.motor_uri = 'mongodb://{account}{hostport0},{hostport1},{hostport2}/{database}? replicaSet=rs0'.format(
        account='{username}:{password}@'.format(
            username=self.MONGODB['MONGO_USERNAME'],
            password=self.MONGODB['MONGO_PASSWORD'])
        if self.MONGODB['MONGO_USERNAME'] else ' ',
        hostport0=self.MONGODB['MONGOHOST_PORT'] [0] if self.MONGODB['MONGOHOST_PORT'] else 'localhost:27017',
        hostport1=self.MONGODB['MONGOHOST_PORT'] [1] if self.MONGODB['MONGOHOST_PORT'] else 'localhost:27016',
        hostport2=self.MONGODB['MONGOHOST_PORT'] [2] if self.MONGODB['MONGOHOST_PORT'] else 'localhost:27015'.# host=self.MONGODB['MONGO_HOST'] if self.MONGODB['MONGO_HOST'] else 'localhost',
        # port=self.MONGODB['MONGO_PORT'] if self.MONGODB['MONGO_PORT'] else 27017,
        database=db)
    print(self.motor_uri)
    return AsyncIOMotorClient(self.motor_uri, retryWrites=False, io_loop=self.loop, tz_aware=True,
                              tzinfo=pytz.timezone('Asia/Shanghai'))
Copy the code

reference

Replica set command

MongoDB is deleted, a replica set is added, and the IP address of the replica set is modified

Add a copy and enter it under Log in to the master node

rs.add("ip:port");
Copy the code

Delete the copy

rs.remove("ip:port");
Copy the code

Adding a Quorum Node

rs.addArb("ip:port");
Copy the code

Alter copy host:

shard1:PRIMARY> cfg = rs.conf()
{
        "_id" : "shard1"."version" : 5,
        "protocolVersion" : NumberLong(1),
        "members": [{"_id": 0."host" : "127.0.0.1:2777"."arbiterOnly" : false."buildIndexes" : true."hidden" : false."priority" : 1,
                        "tags": {},"slaveDelay" : NumberLong(0),
                        "votes": 1}]."settings" : {
                "chainingAllowed" : true."heartbeatIntervalMillis" : 2000,
                "heartbeatTimeoutSecs" : 10,
                "electionTimeoutMillis" : 10000,
                "getLastErrorModes": {},"getLastErrorDefaults" : {
                        "w" : 1,
                        "wtimeout": 0}."replicaSetId" : ObjectId("5d9c7a7e76695600e03e231f")
        }
}


shard1:PRIMARY> cfg.members[0].host = "IP address: 27017"IP address 1:27017 shard1:PRIMARY> rs.reconfig(CFG) {"ok" : 1 }
shard1:PRIMARY> rs.status()
{
        "set" : "shard1"."date" : ISODate("The 2021-06-09 T02:59:26. 916 z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "heartbeatIntervalMillis" : NumberLong(2000),
        "members": [{"_id": 0."name" : "IP address: 27017"."health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY"."uptime" : 54711,
                        "optime" : {
                                "ts" : Timestamp(1570589961, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-06-09T02:59:21Z"),
                        "electionTime" : Timestamp(1570536062, 2),
                        "electionDate" : ISODate("2021-06-09T12:01:02Z"),
                        "configVersion" : 6,
                        "self" : true}]."ok": 1}Copy the code

If your mongo is connected with a replica set, and the master node crashes, your Mongo is not the master node, then it will be dead, here you can force your Mongo node to become the master node. Rs.reconfig (CFG) {“force”: true}).

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Set up Mongo cluster and basic operations

Set up the Mongo cluster

The installation

download

Unpack the

Configuring the Active Node

Configuring slave Nodes

Configuring quorum Nodes

Start the service

master

slave

arbiter

The configuration relationship

The backup data

Configuring Authentication

Mongodump backup

Mongodump common command

Mongorestore Restore practice

Comparison between Mongo backups

Python connects to Mongo as a replica set

Replica set command

Add a copy and enter it under Log in to the master node

Delete the copy

Adding a Quorum Node

Alter copy host:

Set up Mongo cluster and basic operations

Set up the Mongo cluster

The installation

download

Unpack the

Configuring the Active Node

Configuring slave Nodes

Configuring quorum Nodes

Start the service

master

slave

arbiter

The configuration relationship

The backup data

Configuring Authentication

Mongodump backup

Mongodump common command

Mongorestore Restore practice

Comparison between Mongo backups

Python connects to Mongo as a replica set

Replica set command

Add a copy and enter it under Log in to the master node

Delete the copy

Adding a Quorum Node

Alter copy host:

Related Posts

Go Learning Notes (1) – Ubuntu Go environment setup, VS Code Configuration of the Go development environment, VS Code remote development configuration

Command mode for 23 design modes

Is it said that good programmers spend 20% of their time writing UT?