Deploying a High-Availability MongoDB Replication Set on CentOS 7

Deploying a High-Availability MongoDB Replication Set on CentOS 7

Introduction

MongoDB is a high-availability database that uses replication sets to achieve redundancy and failover. In this article, we will describe how to quickly set up a replication set on CentOS 7.

Deploying a Single Node Version

To start, we need to install the MongoDB package on our system. We can do this by adding the MongoDB repository to our yum configuration.

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://repo.mongodb.org/yum/redhat/7Mongodb-org/4.0/x86_64/

Next, we can install the MongoDB package.

sudo yum install -y mongodb-org

This will create two default directories after installation: /var/lib/mongo (data directory) and /var/log/mongodb (log directory).

Starting MongoDB

We can start the MongoDB service using the following commands:

sudo systemctl start mongod.service
sudo systemctl enable mongod.service

Deploying the Replication Set

To deploy the replication set, we need to create three directories: mongo1, mongo2, and mongo3. Each directory should have a mongod.conf file that configures the MongoDB instance.

Configuring the mongod.conf File

The mongod.conf file should contain the following configuration:

systemLog:
  destination: file
  logAppend: true
  path: /home/tenmao/mongo_repl/mongo1/mongod.log

storage:
  dbPath: /home/tenmao/mongo_repl/mongo1/db
  journal:
    enabled: true

processManagement:
  fork: true
  pidFilePath: /home/tenmao/mongo_repl/mongo1/mongod.pid

net:
  port: 27017
  bindIp: 0.0.0.0
  replicaSet:
    replSetName: "tenmao_mongo"

Initializing the Replication Set

To initialize the replication set, we need to start three MongoDB instances with the following commands:

mongod -config mongo1/mongod.conf
mongod -config mongo2/mongod.conf
mongod -config mongo3/mongod.conf

We can then connect to one of the instances and check the replication set status using the following command:

mongo --port 27017
Rs.status ()

This will output the following result:

{
  "Operation": ISODate("2023-07-26T00:00:00.000Z"),
  "Ok": 0,
  "Errmsg": "no replset config has been received",
  "Code": 94,
  "CodeName": "NotYetInitialized",
  "$clusterTime": {
    "clusterTime": ISODate("2023-07-26T00:00:00.000Z"),
    "signature": {
      "hash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId": NumberLong(0)
    }
  }
}

Initializing the Replication Set

To properly initialize the replication set, we need to use the following command:

Rs.initiate ({
  _id: "tenmao_mongo",
  members: [
    { _id: 0, host: "localhost:27017" },
    { _id: 1, host: "localhost:27027" },
    { _id: 2, host: "localhost:27037" }
  ]
})

This will output the following result:

{
  "Ok": 1,
  "Operation": ISODate("2023-07-26T00:00:00.000Z"),
  "$clusterTime": {
    "clusterTime": ISODate("2023-07-26T00:00:00.000Z"),
    "signature": {
      "hash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId": NumberLong(0)
    }
  }
}

Switching to the Master Node

We can switch to the master node using the following command:

use tenmao_mongo
Rs.status ()

This will output the following result:

{
  "stateStr": "PRIMARY",
  "stateNumber": 1,
  "error": "",
  "lastHeartbeat": ISODate("2023-07-26T00:00:00.000Z"),
  "lastHeartbeatRecv": ISODate("2023-07-26T00:00:00.000Z"),
  "optime": ISODate("2023-07-26T00:00:00.000Z"),
  "optimeDate": ISODate("2023-07-26T00:00:00.000Z"),
  "me": {
    "id": 0,
    "name": "localhost:27017",
    "health": 1
  },
  "maxBson": NumberInt(16777216),
  "maxMessageSizeBytes": NumberInt(40000000),
  "maxWireVersion": NumberInt(6),
  "minWireVersion": NumberInt(0),
  "maxDocumentSizeBytes": NumberInt(16777216),
  "roundTripRatio": NumberDouble(1),
  "setVersion": NumberInt(3),
  "setTerm": NumberInt(3),
  "electionId": ObjectId("..."),
  "configVersion": NumberInt(3),
  "hosts": [
    "localhost:27017",
    "localhost:27027",
    "localhost:27037"
  ],
  "hostsReady": [
    "localhost:27017",
    "localhost:27027",
    "localhost:27037"
  ],
  "lastCommittedOp": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastCommittedOpDate": ISODate("2023-07-26T00:00:00.000Z"),
  "lastRemotingOp": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastRemotingOpDate": ISODate("2023-07-26T00:00:00.000Z"),
  "lastVote": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastVoteDate": ISODate("2023-07-26T00:00:00.000Z"),
  "electionDate": ISODate("2023-07-26T00:00:00.000Z"),
  "electionTerm": NumberInt(3),
  "configVersionHash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
  "remoteClientAddress": "localhost:27017",
  "remoteClientAddressIsChecked": true,
  "lastHeartbeatTime": ISODate("2023-07-26T00:00:00.000Z"),
  "lastHeartbeatAddress": "localhost:27017",
  "lastHeartbeatAddressIsChecked": true,
  "lastHeartbeatAddressIsConnected": true,
  "lastHeartbeatAddressIsConnectedIsChecked": true
}

Switching to the Secondary Node

We can switch to the secondary node using the following command:

use tenmao_mongo
Rs.status ()

This will output the following result:

{
  "stateStr": "SECONDARY",
  "stateNumber": 2,
  "error": "",
  "lastHeartbeat": ISODate("2023-07-26T00:00:00.000Z"),
  "lastHeartbeatRecv": ISODate("2023-07-26T00:00:00.000Z"),
  "optime": ISODate("2023-07-26T00:00:00.000Z"),
  "optimeDate": ISODate("2023-07-26T00:00:00.000Z"),
  "me": {
    "id": 1,
    "name": "localhost:27027",
    "health": 1
  },
  "maxBson": NumberInt(16777216),
  "maxMessageSizeBytes": NumberInt(40000000),
  "maxWireVersion": NumberInt(6),
  "minWireVersion": NumberInt(0),
  "maxDocumentSizeBytes": NumberInt(16777216),
  "roundTripRatio": NumberDouble(1),
  "setVersion": NumberInt(3),
  "setTerm": NumberInt(3),
  "electionId": ObjectId("..."),
  "configVersion": NumberInt(3),
  "hosts": [
    "localhost:27017",
    "localhost:27027",
    "localhost:27037"
  ],
  "hostsReady": [
    "localhost:27017",
    "localhost:27027",
    "localhost:27037"
  ],
  "lastCommittedOp": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastCommittedOpDate": ISODate("2023-07-26T00:00:00.000Z"),
  "lastRemotingOp": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastRemotingOpDate": ISODate("2023-07-26T00:00:00.000Z"),
  "lastVote": {
    "ts": ISODate("2023-07-26T00:00:00.000Z"),
    "t": NumberInt(0)
  },
  "lastVoteDate": ISODate("2023-07-26T00:00:00.000Z"),
  "electionDate": ISODate("2023-07-26T00:00:00.000Z"),
  "electionTerm": NumberInt(3),
  "configVersionHash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
  "remoteClientAddress": "localhost:27027",
  "remoteClientAddressIsChecked": true,
  "lastHeartbeatTime": ISODate("2023-07-26T00:00:00.000Z"),
  "lastHeartbeatAddress": "localhost:27027",
  "lastHeartbeatAddressIsChecked": true,
  "lastHeartbeatAddressIsConnected": true,
  "lastHeartbeatAddressIsConnectedIsChecked": true
}

Commonly Used Commands

Here are some commonly used commands for managing the replication set:

  • Rs.status (): Checks the replication set status.
  • Rs.initiate (): Initializes the replication set.
  • Rs.initiate (cfg): Initializes the replication set with a configuration.
  • Rs.conf (): Gets the current configuration object from local.system.replset.
  • Rs.reconfig (cfg): Updates the configuration of a running replica set with cfg.
  • Rs.add (hostportstr): Adds a new member to the set with default attributes.
  • Rs.add (membercfgobj): Adds a new member to the set with extra attributes.
  • Rs.addArb (hostportstr): Adds a new member which is arbiterOnly: true.
  • Rs.stepDown ([stepdownSecs, catchUpSecs]): Steps down as primary.
  • Rs.syncFrom (hostportstr): Makes a secondary sync from the given member.
  • Rs.freeze (secs): Makes a node ineligible to become primary for the specified time.
  • Rs.remove (hostportstr): Removes a host from the replica set.
  • Rs.slaveOk (): Allows queries on secondary nodes.
  • Rs.printReplicationInfo (): Checks oplog size and replica set members.
  • Rs.printSlaveReplicationInfo (): Checks replica set members and replication lag.
  • db.isMaster (): Checks who is primary.

Common Errors

Here are some common errors that may occur:

  • ERROR: child process failed, exited with error number 100: Check the MongoDB log for the problem. In general, this is due to the data directory not existing, which needs to be manually created.
  • not master and slaveOk=false: This is because the default can only read data from the master node, which may be outdated data from the node. You can use Rs.slaveOk () to allow queries on secondary nodes.