Navigation

Deploy a Sharded Cluster

Overview

This tutorial involves creating a new sharded cluster that consists of a mongos, the config server replica set, and two shard replica sets.

For instructions specific to sharding a collection, see Shard a Collection using Hashed Sharding or Shard a Collection using Ranged Sharding.

Considerations

Connectivity

Each member of a sharded cluster must be able to connect to all other members in the cluster. This includes all shards and config servers. Ensure that network and security systems, including all interface and firewalls, allow these connections.

CloudManager and OpsManager

If you are currently using or are planning to use Cloud Manager or Ops Manager, consider using their built-in features for deploying a sharded cluster.

See Deploy a Sharded Cluster in the Cloud Manager manual or in the Ops Manager manual.

Operating System

This tutorial uses the mongod and mongos programs. Windows users should use the mongod.exe and mongos.exe programs instead.

Security

This tutorial does not include the required steps for configuring Internal Authentication or Role-Based Access Control. See Deploy Sharded Cluster with Keyfile Access Control for a tutorial on deploying a sharded cluster with a keyfile.

In production environments, sharded clusters should employ at minimum x.509 security for internal authentication and client access.

For details on using x.509 for internal authentication, see Use x.509 Certificate for Membership Authentication.

For details on using x.509 for client authentication, see Use x.509 Certificates to Authenticate Clients.

Note

Enabling internal authentication also enables Role-Based Access Control.

Host Identifier

If you use either localhost or 127.0.0.1 as the hostname portion of any host identifier, you must use that identifier as the host setting for any other MongoDB component in the cluster.

For example, the sh.addShard() method takes a host parameter for the hostname of the target shard. If you set host to localhost, you must then use localhost as the host for all other shards in the cluster.

Deploy Sharded Cluster

Create the Config Server Replica Set

The following steps deploys a config server replica set.

For a production deployment, deploy a config server replica set with at least three members. For testing purposes, you can create a single-member replica set.

Note

The config server replica set must not use the same name as any of the shard replica sets.

1

Start each member of the config server replica set.

When starting each mongod, specify the mongod settings either via a configuration file or the command line.

Configuration File

If using a configuration file, set:

sharding:
  clusterRole: configsvr
replication:
  replSetName: <replica set name>
net:
  bindIp: localhost,<ip address>

Start the mongod with the --config option set to the configuration file path.

mongod --config <path-to-config-file>

Command Line

If using the command line options, start the mongod with the --configsvr, --replSet, --bind_ip, and other options as appropriate to your deployment. For example:

Warning

Before you bind to other ip addresses, consider enabling access control and other security measures listed in Security Checklist to prevent unauthorized access.

mongod --configsvr --replSet <replica set name> --dbpath <path> --bind_ip localhost,<ip address of the mongod host>

For more information on startup parameters, see the mongod reference page.

2

Connect to one of the config servers.

Connect a mongo shell to one of the config server members.

mongo --host <hostname> --port <port>
3

Initiate the replica set.

From the mongo shell, run the rs.initiate() method.

rs.initiate() can take an optional replica set configuration document. In the replica set configuration document, include:

  • The _id set to the replica set name specified in either the replication.replSetName or the --replSet option.
  • The configsvr field set to true for the config server replica set.
  • The members array with a document per each member of the replica set.

Important

Run rs.initiate() on just one and only one mongod instance for the replica set.

rs.initiate(
  {
    _id: "<replSetName>",
    configsvr: true,
    members: [
      { _id : 0, host : "cfg1.example.net:27019" },
      { _id : 1, host : "cfg2.example.net:27019" },
      { _id : 2, host : "cfg3.example.net:27019" }
    ]
  }
)

See Replica Set Configuration for more information on replica set configuration documents.

Once the config server replica set (CSRS) is initiated and up, proceed to creating the shard replica sets.

Create the Shard Replica Sets

For a production deployment, use a replica set with at least three members. For testing purposes, you can create a single-member replica set.

Note

Shard replica sets must not use the same name as the config server replica set.

1

Start each member of the shard replica set.

When starting each mongod, specify the mongod settings either via a configuration file or the command line.

Configuration File

If using a configuration file, set:

sharding:
   clusterRole: shardsvr
replication:
   replSetName: <replSetName>
net:
   bindIp: localhost,<ip address>

Start the mongod with the --config option set to the configuration file path.

mongod --config <path-to-config-file>

Command Line

If using the command line option, start the mongod with the --replSet, and --shardsvr, --bind_ip options, and other options as appropriate to your deployment. For example:

mongod --shardsvr --replSet <replSetname>  --dbpath <path> --bind_ip localhost,<ip address of the mongod host>

For more information on startup parameters, see the mongod reference page.

2

Connect to one member of the shard replica set.

Connect a mongo shell to one of the replica set members.

mongo --host <hostname> --port <port>
3

Initiate the replica set.

From the mongo shell, run the rs.initiate() method.

rs.initiate() can take an optional replica set configuration document. In the replica set configuration document, include:

  • The _id field set to the replica set name specified in either the replication.replSetName or the --replSet option.
  • The members array with a document per each member of the replica set.

The following example initiates a three member replica set.

Important

Run rs.initiate() on just one and only one mongod instance for the replica set.

rs.initiate(
  {
    _id : <replicaSetName>,
    members: [
      { _id : 0, host : "s1-mongo1.example.net:27018" },
      { _id : 1, host : "s1-mongo2.example.net:27018" },
      { _id : 2, host : "s1-mongo3.example.net:27018" }
    ]
  }
)

Connect a mongos to the Sharded Cluster

1

Connect a mongos to the cluster

Start a mongos using either a configuration file or a command line parameter to specify the config servers.

Configuration File

If using a configuration file, set the sharding.configDB to the config server replica set name and at least one member of the replica set in <replSetName>/<host:port> format.

Warning

Before you bind to other ip addresses, consider enabling access control and other security measures listed in Security Checklist to prevent unauthorized access.

sharding:
  configDB: <configReplSetName>/cfg1.example.net:27019,cfg2.example.net:27019
net:
  bindIp: localhost,<ip address>

Start the mongos specifying the --config option and the path to the configuration file.

mongos --config <path-to-config>

For more information on the configuration file, see configuration options.

Command Line

If using command line parameters start the mongos and specify the --configdb, --bind_ip, and other options as appropriate to your deployment. For example:

Warning

Before you bind to other ip addresses, consider enabling access control and other security measures listed in Security Checklist to prevent unauthorized access.

mongos --configdb <configReplSetName>/cfg1.example.net:27019,cfg2.example.net:27019 --bind_ip localhost,<ip address of the mongos host>

Include any other options as appropriate for your deployment.

2

Connect to the mongos.

Connect a mongo shell to the mongos.

mongo --host <hostname> --port <port>

Add Shards to the Cluster

Use the sh.addShard() method to add each shard to the cluster. If the shard is a replica set, specify the name of the replica set and specify a member of the set. In production deployments, all shards should be replica sets.

The following operation adds a single shard replica set to the cluster:

sh.addShard( "<replSetName>/s1-mongo1.example.net:27017")

The following operation is an example of adding a standalone mongod shard to the cluster:

sh.addShard( "s1-mongo1.example.net:27017")

Repeat these steps until the cluster includes all shards.

Enable Sharding for a Database

To proceed, you must be connected to a mongos associated to the target sharded cluster.

Before you can shard a collection, you must enable sharding for the collection’s database. Enabling sharding for a database does not redistribute data but make it possible to shard the collections in that database.

Once you enable sharding for a database, MongoDB assigns a primary shard for that database where MongoDB stores all data in that database.

Use the sh.enableSharding() method to enable sharding on the target database.

sh.enableSharding("<database>")

Shard a Collection

This section contains an overall description of the sharding process.

For instructions specific to Ranged Sharding sharding, see Shard a Collection using Ranged Sharding.

For instructions specific to Hashed Sharding sharding, see Shard a Collection using Hashed Sharding

To proceed, you must be connected to a mongos associated to the target sharded cluster.

To shard a collection, use the sh.shardCollection() method. You must specify the full namespace of the collection and a document containing the shard key. The database must have sharding enabled.

Your selection of shard key affects the efficiency of sharding, as well as your ability to take advantage of certain sharding features such as zones. See the selection considerations listed in the Choosing a Shard Key.

If the collection already contains data, you must create an index on the shard key using the db.collection.createIndex() method before using shardCollection().

If the collection is empty, MongoDB creates the index as part of sh.shardCollection().

The following operation shards the target collection:

sh.shardCollection("<database>.<collection>", { <key> : <direction> } )