Introduction to the

NOSQL has these advantages:

  1. Large amount of data, you can store a large amount of data through cheap servers, easily get rid of the traditional mysql single table storage limit.
  2. High scalability, Nosql removed the relational database relational features, it is easy to horizontal expansion, get rid of the past always vertical expansion.
  3. High performance. Nosql uses a simple key-value method to get data quickly. Also, NoSQL’s Cache is record level, which is a fine-grained Cache, so NoSQL is much better at this level.
  4. Flexible data model, NoSQL does not need to create fields for the data to be stored, and can store custom data formats at any time. In relational databases, adding and deleting fields is a very troublesome thing. If you have a very large table, adding fields is a nightmare.
  5. Highly available, NoSQL can easily implement highly available architectures without much impact on performance. For example, mongodb can quickly configure high availability configurations through Mongos and Mongo sharding.
  6. Supports query, aggregation, and full indexing, including internal objects
  7. Supports replication, failover, and automatic recovery
  8. Easy extension

In the NOSQL database, most queries are in the form of key and value pairs. MongoDB is a product between relational database and non-relational database, the most like relational database among non-relational databases. Support similar to object-oriented query language, can achieve almost the majority of functions similar to relational database single table query, but also support to index data. So this is very convenient, we can use SQL to operate MongoDB, from the relational database migration, developer learning costs will be greatly reduced. If you add a layer of encapsulation to the underlying SQL API, developers can hardly feel the difference between mongodb and relational databases.

MongoDB is a database based on distributed file storage. Written in C++ language; Designed to provide scalable high-performance data storage solutions for WEB applications.

Install the mongo

Installation environment

Operating system: Centos7.2

Mongodb version: V3.6.1

Download and install

Wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-amazon-3.6.1.tgz tar ZXVF Mongo - Linux - x86_64 - amazon - 3.6.1. TGZ mv/root/mongo - Linux - x86_64 - amazon - 3.6.1 track/usr/local/mongo /Copy the code

Create a data/log directory

mkdir -p /data/mongodb/{data, logs}

mkdir /data/mongodb/data/mongod
touch /data/mongodb/logs/mongo.logsCopy the code

Creating a Configuration File

mkdir /usr/local/mongodb/config
cd  /usr/local/mongodb/config && touch mongo.confCopy the code

The configuration file

1. Example of a common configuration file

Dbpath = / data/mongo/data/mongod logpath = / data/mongo/logs/mongo. Log logappend = true replSet bind_ip = = mongo - rs 0.0.0.0 port=27017 fork=true journal=trueCopy the code

Mongodb3. After x version is yaml syntax format of configuration files, the following is yaml configuration file format is as follows: the official yaml configuration file options reference: docs.mongodb.org/manual/refe… Note: Only Spaces can be used. TAB is not supported

2. Example of a configuration file in yamL format

storage:
    dbPath: /data/mongodb/data/mongod
    journal:
        enabled: true
systemLog:
    destination: file
    path: /data/mongodb/logs/mongo.log
    logAppend: true
    logRotate: rename
net:
    bindIp: 0.0.0.0
    port: 27017
processManagement:
    pidFilePath: /var/run/pid/mongodb.pid
    fork: true
replication:
    oplogSizeMB: 20480
    replSetName: mongo-rsCopy the code

Description of configuration file parameters

1. Basic parameters

--quiet --port arg # specifies the service port number. The default port number is 27017. --bind_ip arg # specifies the service IP address. -- logPath arg # specify MongoDB log files -- logAppend # append -- pidFilepath arg # PID full path to File, If not set, there is no PID file --keyFile arg # cluster private key full path, Only valid for Replica Set --unixSocketPrefix arg # UNIX domain socket replacement directory (default: / TMP) --fork # run MongoDB as a daemon, Create server process --auth # enable validation -- CPU # display CPU utilization and IOwait periodically -- dbPath arg # specify database path --diaglog arg # diaglog option 0=off 1=W 2=R 3=both 7=W+some reads --directoryperdb # set each database to be saved in a separate directory --journal # enable the logging option, --journalOptions arg # enable log diagnostics --ipv6 # enable ipv6 -- jSONp # Allow JSONP access over HTTP (security implications) --maxConns arg # Default maximum number of simultaneous connections 2000 --noauth # disable authentication -- nohttpInterface # disable HTTP interface Disable port 27018 access by default -- NoPrealloc # Disable data file preallocation (often affecting performance) -- Noscripting # Disable scripting engine -- NotablesCAN # Do not allow table scanning -- Nounixsocket # Disable Unix socket listening --nssize arg (=16) # set message database.ns file size (MB) -- objCheck # Check the validity of receiving client data, --profile arg # file parameter 0=off 1=slow, 2=all --quotaFiles arg # number of files allower per db, Repairpath requires -- Quota -- REST # Start the simple REST API --repair # Run repair on all DBS -- RepairPath ARG # Dbpath --slowms arg (=100) # value of slow for profile and console log --smallfiles # use smaller default files -- Syncdelay arg (=60) # Number of seconds for data to be written to disk (0=never, not recommended) --sysinfo # Print some diagnostic system information --upgrade # Upgrade database if necessaryCopy the code

2. Replicaton parameters

-- Fastsync # Enable the slave database replication service from a DBPATH that is a snapshot of the master database and can be used to quickly enable synchronization. --oplogSize arg # Set oplog size (MB)Copy the code

3. Master/slave parameters

--master # Master database mode --slave # slave database mode --source arg # slave database port number --only arg # specifies a single database replication --slavedelay arg # Sets the delay time for synchronizing the master database from the slave databaseCopy the code

4.Replica Set option

--replSet arg # set the replica set name Sharding option -- configSvr # declare this to be the config service for a cluster, default port 27019, Default directory /data/configdb -- shardSvr # declares this to be a sharding of a cluster, default port 27018 --noMoveParanoia # closes for moveChunk data savingCopy the code

Start the

mongod --quiet -f /usr/local/mongodb/config/mongo.confCopy the code

Fork :true = ‘fork’ = ‘logpath’ = ‘fork’ = ‘logpath’

mongod --dbpath=/data/mongodb/data/mongod --fork --logpath=/data/mongodb/logs/mongo.logsCopy the code

The cluster structures,

It is not recommended to use the master/slave cluster mode. The recommended cluster mode is Replica Set (Replica Set). In fact, the master/slave cluster mode is a single copy application, which does not have good scalability and fault tolerance. However, the replica set has multiple copies to ensure fault tolerance. Even if one copy fails, many copies still exist, and solves the first problem above: “When the master node fails, the whole cluster will automatically switch”.

The design structure of replica sets

You can see from the diagram that the client is connected to the entire replica set, regardless of which machine is down or not. The primary server is responsible for reading and writing the entire replica set, and the replica set periodically synchronizes data backups. Once the primary node fails, the replica node elects a new primary server, which is not a concern for the application server.

Note:

A quorum node is a special node that does not store data. Its main function is to determine which secondary node is promoted to the primary node after the primary node fails. Therefore, clients do not need to connect to this node. There is only one standby node, but a quorum node is still needed to upgrade the standby node.

There must be a quorum node. If there is no quorum node, the standby node is still the standby node.

The configuration steps

Prepare three machines

172.29.142.17 Active 172.29.142.18 Standby 172.28.226.199 ArbitrationCopy the code

Install and start on the three machines as per step 2 installation

/usr/local/mongodb/bin/mongod --quiet -f /usr/local/mongodb/config/mongo.confCopy the code

Initialize the cluster configuration

Starting three services does not mean they are in a cluster, so the cluster needs to be initialized. To connect any node (not a quorum point), do the following:

Rs. initiate({_id:" mongo-RS ", # cluster name members:[{_id:0,host:'172.29.142.18:27017',priority:2}, # main {_id: 1, host: '172.29.142.17:27017' priority: 1}, # for {_id: 2, host: '172.28.226.199:27017, arbiterOnly: true}}] # arbitration node)Copy the code

OK is returned on success and the cluster status is checked, which is done on the standby node

rs.status()Copy the code

Returns the name and members information for the cluster, as in:

{" set ", "mongo - rs", "date" : ISODate (" the 2018-06-26 T14:56:08. 032 z "), "myState" : 2, the "term" : NumberLong(2), "syncingTo" : "172.29.142.18:27017", "heartbeatIntervalMillis" : NumberLong(2000), "opTimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1530024958, 1), "t" : NumberLong(2) }, "appliedOpTime" : { "ts" : Timestamp(1530024958, 1), "t" : NumberLong(2) }, "durableOpTime" : { "ts" : Timestamp(1530024958, 1), "t" : NumberLong (2)}}, "members" : [{" _id ": 0," name ":" 172.29.142.18:27017 ", "health" : 1.0, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 382251, "optime" : { "ts" : Timestamp(1530024958, 1), "t" : NumberLong(2) }, "optimeDurable" : { "ts" : Timestamp(1530024958, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate(" 2018-06-26T14:55:58.000z "), "optimeDurableDate" : ISODate(" 2018-06-26T14:55:58.000z "), "lastHeartbeat" : PingMs: ISODate(" 2018-06-26T14:56:07.329z "), "lastHeartbeatRecv" : ISODate(" 2018-06-26T14:56:06.453z "), "pingMs" : NumberLong(0), "electionTime" : Timestamp(1529642739, 1), "electionDate" : ISODate(" 2018-06-22T04:45:39.000z "), "configVersion" : 1}, {"_id" : 1, "name" : "172.29.142.17:27017", "health" : 1.0, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 382552, "optime" : {"ts" : Timestamp(1530024958, 1), "t" : NumberLong(2)}, "optimeDate" : ISODate(" 2018-06-26T14:55:58.000z "), "syncingTo" : "172.29.142.18:27017", "configVersion" : 1, "self" : true}, {"_id" :2, "name" : "172.28.226.1990:27017 ", "health" : 1.0, "state" : 7, "stateStr" : "ARBITER", "uptime" : 168617, "lastHeartbeat" : ISODate(" 2018-06-26T14:56:06.895z "), "lastHeartbeatRecv" : ISODate(" 2018-06-26T14:56:04.092z "), "pingMs" : NumberLong(35), "configVersion" : 1}], "OK" : 1.0}Copy the code

Return Parameter Description

“Health” : 1, # indicates that the machine is normal. “stteStr” : “PRIMARY”, # indicates that the PRIMARY node is readable and writable.

STARTUP: newly added to the replication set, and the configuration is not loaded. STARTUP2: The configuration is loaded, and the initialization state is RECOVERING. Arbiter DOWN: the node is unreachable UNKNOWN: the state of the node is UNKNOWN because it has not been obtained. This occurs in an architecture with only two members. Data rollback, at the end of the rollback, shifts to the RECOVERING or SECONDARY FATAL: error. Run the grep replSet FATAL command to find the error cause and synchronize data again. PRIMARY: indicates the PRIMARY node. SECONDARY: indicates the backup nodeCopy the code

Test replica set data replication

Note: bY default, mongodb reads and writes data from the master node. Read data is not allowed on the replica node. You need to set the replica node to read data:

repset:SECONDARY> db.getMongo().setSlaveOk();Copy the code

This is easy to test, simply insert a piece of data on the primary node and query it on the secondary node

Or you can use the client to connect to the Mongo cluster in cluster mode:

Click Test to Test connection:

Data on all three nodes is synchronized.

Test replica set failover functionality

1. View the current cluster status

Currently, 172.29.142.18 is Primary and 172.29.142.17 is Secondary

 

2. Disable 172.29.142.18 and view the results of the other two nodes

At this point, 17 becomes the master node, and the original quorum node remains unchanged. Restart the Primary node for the first time, and the master node changes again. No screenshots are taken, and the whole process is uninterrupted. As long as one is available.

Nodejs connects to the Mongo cluster example

Connecting to a single Mongo service is not strongly recommended here, because if a Mongo node fails, the business dies, and connecting to the cluster only requires that one be available.

Here is an example of a nodeJS connection to a Mongo cluster:

const mongoose = require('mongoose'); let url = "Mongo: / / 172.29.142.17:27017 / testdb, mongo: / / 172.29.142.18:27017 / testdb, mongo: / / 172.28.226.199:27017 / testdb"; let options = { "replset": { "ha": true, "haInterval": 1000, "replicaSet": "mongo-rs", "connectWithNoPrimary": true, "auto_reconnect": true, "socketOptions": { "keepAlive": 120, connectTimeoutMS: Connect (url, options).connection. on('error', function (error) {console.log('mongo connection error', Error)}). On ('open', mongoConnect). Once ('open', function () {console.log('mongo connection successful '); })Copy the code

 

reference:

Blog.csdn.net/luonanqin/a…

Blog.csdn.net/wangshuang1…

zhoujie
www.cnblogs.com/zhoujie/