MongoDB development replication-set how-to

I will describe how to configure and run a MongoDB (v2.0.6) replication set consisting of 3 nodes on localhost machine. For more informations (architecture, voting, etc.) please refer to http://www.mongodb.org/display/DOCS/Replica+Sets

1. Create necessary directories for each node

sudo mkdir -p /srv/mongodb/rs0-0
sudo mkdir -p /srv/mongodb/rs0-1
sudo mkdir -p /srv/mongodb/rs0-2

2. Create configuration files for each node

sudo touch /srv/mongodb/rs0-0/node0.conf
sudo touch /srv/mongodb/rs0-1/node1.conf
sudo touch /srv/mongodb/rs0-2/node2.conf

3. Edit each configuration file respectively:

3.1 node0.conf
# mongo.node0.conf
replSet=r0
logpath=node0.log
port = 27017
logappend=true
dbpath=/srv/mongodb/rs0-0
fork = true
rest = true

3.2 node1.conf
# mongo.node1.conf
replSet=r0
logpath=node1.log
port = 27018
logappend=true
dbpath=/srv/mongodb/rs0-1
fork = true
rest = true

3.3 node2.conf
# mongo.node2.conf
replSet=r0
logpath=node2.log
port = 27019
logappend=true
dbpath=/srv/mongodb/rs0-2
fork = true
rest = true

We are creating 3 nodes, each of them works on different port (27017, 27018, 27019), in the same replication set (r0), as a process (fork) with enabled logging and rest interface.

4. Create configuration for replication set

When initalizing a replication set all machines need to know about each others. I would create a text file in json style in main directory and later on copy it's contents to mongo console.

sudo touch /srv/mongodb/config

config = {_id: "r0", 
        members:[
                {_id: 0, host: '127.0.0.1:27017'},
                {_id: 1, host: '127.0.0.1:27018'},
                {_id: 2, host: '127.0.0.1:27019', priority: 0, slaveDelay: 60},
                ]
};

We are creating a new set (named r0) with an array of members. Notice that machine with id 2 works with delay (each operation will be executed with 60 seconds delay) therefore it cannot be elected as primary (priority to zero, necessary condition).

5. Run all nodes

Open 4 terminal windows. First three will represent nodes, the last one will be a mongo client used for establishing connections.

Terminals 1-3 (in directory /srv/mongodb/rs0-x):

sudo mongod -f node[x].conf
tail -f node[x].log

(where [x] stands for node number)

6. Connect to any node and initialize a replication set

In the last terminal type:

mongo localhost:27018 (port could be 27017 or 27019, it doesn't matter right now)

Copy and paste the contents of /srv/mongodb/config file formerly created - it will now become a json-style variable.

Then in console type:

> rs.initiate(config);

As an output you ought see:

{
"info" : "Config now saved locally.  Should come online in about a minute.",
"ok" : 1
}

From now, nodes started communicating to each other. You can see current state of the set by typing:

> rs.status()

{
"set" : "r0",
"date" : ISODate("2012-10-30T15:30:20Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "127.0.0.1:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 109,
"optime" : {
"t" : 1351610907000,
"i" : 1
},
"optimeDate" : ISODate("2012-10-30T15:28:27Z"),
"lastHeartbeat" : ISODate("2012-10-30T15:30:19Z"),
"pingMs" : 0
},
{
"_id" : 1,
"name" : "127.0.0.1:27018",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"optime" : {
"t" : 1351610907000,
"i" : 1
},
"optimeDate" : ISODate("2012-10-30T15:28:27Z"),
"self" : true
},
{
"_id" : 2,
"name" : "127.0.0.1:27019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 111,
"optime" : {
"t" : 1351610907000,
"i" : 1
},
"optimeDate" : ISODate("2012-10-30T15:28:27Z"),
"lastHeartbeat" : ISODate("2012-10-30T15:30:19Z"),
"pingMs" : 0
}
],
"ok" : 1
}

We can see that all machines are up and in good condition. Node with id was chosen as a PRIMARY node (write/read permissions), while the others are in SECONDARY state.

7. Extras

7.1 Experiments
Experiment how nodes behave in certain situations:
- insert data into primary node, reconnect to other machine and check if it is still available
- insert data on secondary machine (tip. use slaveOk function) and check if it is available to other nodes
- break one node (ie. a safe way to disconnect slave node is to connect to that node and type

use admin
db.shutdownServer()

then reconnect to your primary node, load some data, after a while bring broken node back to live and try to observe the process of automatic recovery (rs.status())
- insert a large set of data, disconnect one node, ensure an index on some field (might take time), reconnect this node as a primary, than perform recovery

7.2 Web status
You can check current status of your set using web browser. Just type http://localhost:28017/_replSet

7.3 Production
Remeber that this configuration is for learning puroposes only. When using it on production always assure that each node is on different machine (or in cloud)