Navigation

Perform Maintenance on Replica Set Members

On this page

Overview

Replica sets allow a MongoDB deployment to remain available during the majority of a maintenance window.

This document outlines the basic procedure for performing maintenance on each of the members of a replica set. Furthermore, this particular sequence strives to minimize the amount of time that the primary is unavailable and controlling the impact on the entire deployment.

Use these steps as the basis for common replica set operations, particularly for procedures such as upgrading to the latest version of MongoDB and changing the size of the oplog.

Procedure

For each member of a replica set, starting with a secondary member, perform the following sequence of events, ending with the primary:

  • Restart the mongod instance as a standalone.
  • Perform the task on the standalone instance.
  • Restart the mongod instance as a member of the replica set.
1

Stop a secondary.

In the mongo shell, shut down the mongod instance:

db.shutdownServer()
2

Restart the secondary as a standalone on a different port.

At the operating system shell prompt restart mongod as a standalone instance.

If you are using a configuration file, make the following configuration updates:

For example, if performing maintenance on a shard/config server replica set member for maintenance, the updated configuration file will include content like the following example:

net:
   port: 27218
#   port: 27018
#replication:
#   replSetName: shardA
#sharding:
#   clusterRole: shardsvr
setParameter:
   skipShardingConfigurationChecks: true

If using command-line options, make the following configuration updates to restart:

For example, to restart a replica set member that is not part of a sharded cluster:

mongod --port 27218 --dbpath /srv/mongodb

For example, to restart a shard/config server replica set member for maintenance:

mongod --port 27218 --dbpath /srv/mongodb  --setParameter skipShardingConfigurationChecks=true

Always start mongod with the same user, even when restarting a replica set member as a standalone instance.

3

Perform maintenance operations on the secondary.

While the member is a standalone, use the mongo shell to perform maintenance:

mongo --port 27218
4

Restart mongod as a member of the replica set.

After performing all maintenance tasks, use the following procedure to restart the mongod as a member of the replica set on its usual port.

From the mongo shell, shut down the standalone server after completing the maintenance:

db.shutdownServer()

Restart the mongod instance as a replica set member with its original configuration; that is, undo the configuration changes made when starting as a standalone.

For shard or config server members, be sure to remove the skipShardingConfigurationChecks parameter.

The secondary takes time to catch up to the primary. From the mongo shell, use the following command to verify that the member has caught up from the RECOVERING state to the SECONDARY state.

rs.status()
5

Perform maintenance on the primary last.

To perform maintenance on the primary after completing maintenance tasks on all secondaries, use rs.stepDown() in the mongo shell to step down the primary and allow one of the secondaries to be elected the new primary. Specify a 300 second waiting period to prevent the member from being elected primary again for five minutes:

rs.stepDown(300)

After the primary steps down, the replica set will elect a new primary.

Restart mongod as a standalone instance, making the following configuration updates.

If you are using a configuration file, make the following configuration updates:

For example, if performing maintenance on a shard/config server replica set member for maintenance, the updated configuration file will include content like the following example:

net:
   port: 27218
#   port: 27018
#replication:
#   replSetName: shardA
#sharding:
#   clusterRole: shardsvr
setParameter:
   skipShardingConfigurationChecks: true

If using command-line options, make the following configuration updates:

For example, to restart a replica set member that is not part of a sharded cluster:

mongod --port 27218 --dbpath /srv/mongodb

For example, to restart a shard/config server replica set member for maintenance:

mongod --port 27218 --dbpath /srv/mongodb  --setParameter skipShardingConfigurationChecks=true

After performing all maintenance tasks, restart the mongod instance as a replica set member with its original configuration; that is, undo the configuration changes made when starting as a standalone.

For shard or config server replica set members, be sure to remove the skipShardingConfigurationChecks parameter.