Navigation

Deploy a Replica Set

Cloud Manager and Ops Manager 4.0.11 Support MongoDB Resources

You can use the Kubernetes Operator to deploy MongoDB resources with Ops Manager version 4.0.11 or later and Cloud Manager. At any place in this guide that says Ops Manager, you can substitute Cloud Manager.

A replica set is a group of MongoDB deployments that maintain the same data set. Replica sets provide redundancy and high availability and are the basis for all production deployments.

To learn more about replica sets, see the Replication Introduction in the MongoDB manual.

Use this procedure to deploy a new replica set that Ops Manager manages. After deployment, use Ops Manager to manage the replica set, including such operations as adding, removing, and reconfiguring members.

Prerequisites

To deploy a replica set using an object, you need to complete the following procedures:

Considerations

Starting in MongoDB Enterprise Kubernetes Operator version 1.3.0, you can only have one MongoDB resource per project. To learn how to migrate your project to a single-cluster configuration, see Migrate to One Resource per Project (Required for Version 1.3.0).

Deploy a Replica Set

1

Configure kubectl to default to your namespace.

If you have not already, run the following command to execute all kubectl commands in the namespace you created:

kubectl config set-context $(kubectl config current-context) --namespace=<namespace>
2

Copy the highlighted section of this replica set resource.

Change the highlighted settings of this YAML file to match your desired replica set configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
---
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: <my-replica-set>
spec:
  members: 3
  version: 4.2.2-ent
  opsManager:
    configMapRef:
      name: <configMap.metadata.name>
            # Must match metadata.name in ConfigMap file
  credentials: <mycredentials>
  type: ReplicaSet
  persistent: true
...
3

Paste the copied example to create a new replica set resource.

Open your preferred text editor and paste the object specification into a new text file.

4

Change the highlighted settings to your preferred values.

Key Type Description Example
metadata.name string

Label for this Kubernetes replica set object.

Resource names must be 44 characters or less.

See also

myproject
spec.members integer Number of members of the replica set. 3
spec.version string

Version of MongoDB that this replica set should run.

The format should be X.Y.Z for the Community edition and X.Y.Z-ent for the Enterprise edition.

To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.

3.6.7
spec
.opsManager
.configMapRef
string

Name of the ConfigMap with the Ops Manager connection configuration. The spec.cloudManager.configMapRef.name setting is an alias for this setting and can be used in its place.

Value must match namespace and name of ConfigMap

This value must match the namespace in which you created the Ops Manager project ConfigMap.

If this MongoDB Kubernetes resource is in a different namespace than the project ConfigMap, you should set this value to the namespace and name of the ConfigMap in this format: <metadata.namespace>/<metadata.name>

Operator manages changes to the ConfigMap

The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the MongoDB Kubernetes resource.

<myconfigmap> or <namespace>/<myconfigmap>
spec.credentials string

Name of the Kubernetes secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager.

Value must use namespace and name of Secret

This value must match the namespace in which you created the secret and the name value you provided for your Ops Manager Kubernetes Secret.

If this object is in a different namespace than the Secret, you should set this value to the namespace and name of the Secret in this format: <namespace>/<name>

Operator manages changes to the Secret

The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the MongoDB Kubernetes resource.

<mycredentials> or <namespace>/<mycredentials>
spec.type string Type of MongoDB Kubernetes resource to create. ReplicaSet
spec.persistent string

Optional.

Flag indicating if this MongoDB Kubernetes resource should use Persistent Volumes for storage. Persistent volumes are not deleted when the MongoDB Kubernetes resource is stopped or restarted.

If this value is true, then spec.podSpec.persistence.single is set to its default value of 16G.

To change your Persistent Volume Claims configuration, configure the following collections to meet your deployment requirements:

Warning

Your containers must have permissions to write to your Persistent Volume. The Kubernetes Operator sets fsGroup = 2000 in securityContext This makes Kubernetes try to fix write permissions for the Persistent Volume. If redeploying the deployment item does not fix issues with your Persistent Volumes, contact MongoDB Support.

Note

If you do not use Persistent Volumes, the Disk Usage and Disk IOPS charts cannot be displayed in either the Processes tab on the Deployment page or in the Metrics page when reviewing the data for this deployment.

true
5

Add any additional accepted settings for a replica set deployment.

You can also add any of the following optional settings to the object specification file for a replica set deployment:

Warning

You must set spec.clusterDomain if your Kubernetes cluster has a default domain other than the default cluster.local. If you neither use the default nor set the spec.clusterDomain option, the Kubernetes Operator might not function as expected.

6

Save this replica set config file with a .yaml file extension.

7

Start your replica set deployment.

Invoke the following Kubernetes command to create your replica set:

kubectl apply -f <replica-set-conf>.yaml
8

Track the status of your replica set deployment.

To check the status of your MongoDB Kubernetes resource, invoke the following command:

kubectl get mdb <resource-name> -o yaml -w

The -w flag means “watch”. With the “watch” flag set, the output refreshes immediately when something changes until the status phase achieves the Running state.

See Troubleshooting the Kubernetes Operator for information about the resource deployment statuses.

Enable External Access for a Replica Set

1

If you haven’t done so already, deploy a replica with the Kubernetes Operator.

Follow the instructions to :ref:` deploy a replica set <deploy-replica-set>`. To simplify the configuration, don’t enable TLS with the spec.security.tls.enabled setting.

2

Optional: If you already deployed a replica set with the Kubernetes Operator with TLS enabled, remove the CSRs for each host in your deployment.

  1. Invoke the following command to retrieve the CSRs for each host:

    kubectl get csr
    

    The command’s output resembles the following:

    NAME                                        AGE       REQUESTOR                                                   CONDITION
    my-secure-rs-0.mongodb                      33s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved, Issued
    my-secure-rs-1.mongodb                      31s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved, Issued
    my-secure-rs-2.mongodb                      24s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Approved, Issued
    
  2. Repeat the following command for each host in your deployment to remove the CSRs:

    kubectl delete my-secure-rs-0.mongodb
    

    Important

    Remove only the TLS CSRs. Don’t remove X.509 or any other CSRs.

3

Create a NodePort for each pod.

Invoke the following commands to create the NodePorts:

kubectl expose pod/<my-replica-set>-0 --type="NodePort" --port 27017
kubectl expose pod/<my-replica-set>-1 --type="NodePort" --port 27017
kubectl expose pod/<my-replica-set>-2 --type="NodePort" --port 27017
4

Discover the dynamically assigned NodePorts.

Discover the dynamically assigned NodePorts:

$ kubectl get svc | grep <my-replica-set>
<my-replica-set>-0                      NodePort    172.30.39.228    <none>        27017:30907/TCP              16m
<my-replica-set>-1                      NodePort    172.30.185.136   <none>        27017:32350/TCP              16m
<my-replica-set>-2                      NodePort    172.30.84.192    <none>        27017:31185/TCP              17m
<my-replica-set>-svc                    ClusterIP   None             <none>        27017/TCP                    38m

NodePorts range from 30000 to 32767, inclusive.

5

Open your replica set resource YAML file.

6

Copy the highlighted section of this replica set resource.

Change the highlighted settings of this YAML file to match your desired replica set configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
---
apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: <my-replica-set>
spec:
  members: 3
  version: 4.2.2-ent
  type: ReplicaSet
  opsManager:
    configMapRef:
      name: <configMap.metadata.name>
  credentials: <mycredentials>
  persistent: true
15
16
17
18
19
20
21
22
23
  security:
    tls:
      enabled: true
  connectivity:
    replicaSetHorizons:
      - "example-website": "web1.example.com:10017"
      - "example-website": "web2.example.com:10017"
      - "example-website": "web3.example.com:10017"
...
7

Paste the copied example section into your existing replica set resource.

Open your preferred text editor and paste the object specification at the end of your resource file in the spec section.

8

Change the highlighted settings to your preferred values.

Key Type Necessity Description Example
spec.security.tls
boolean Optional

Set this value to true to enable TLS on the MongoDB deployment.

By default, Kubernetes Operator requires hosts to use and accept TLS encrypted connections.

Note

To connect to a replica set from outside Kubernetes, set this value to true.

true
spec.connectivity
collection Conditional

Add this parameter and values if you need your database to be accessed outside of Kubernetes. This setting allows you to provide different DNS settings within the Kubernetes cluster and to the Kubernetes cluster. The Kubernetes Operator uses split horizon DNS for replica set members. This feature allows communication both within the Kubernetes cluster and from outside Kubernetes.

You may add multiple external mappings per host.

Split Horizon Requirements

  • Make sure that each value in this array is unique.
  • Make sure that the number of entries in this array matches the value given in spec.members.
  • Set the spec.security.tls.enabled to true to enable TLS. This method to use split horizons requires the Server Name Indication extension of the TLS protocol.
See Setting
9

Confirm the external hostnames and NodePort values in your replica set resource.

Confirm that the external hostnames in the spec.connectivity.replicaSetHorizons setting are correct.

External hostnames should match the DNS names of Kubernetes worker nodes. These can be any nodes in the Kubernetes cluster. nodes do internal routing if the pod is run on another node.

Set the ports in spec.connectivity.replicaSetHorizons to the NodePort values that you discovered.

Example

15
16
17
18
19
20
21
22
23
  security:
    tls:
      enabled: true
  connectivity:
    replicaSetHorizons:
      - "example-website": "web1.example.com:30907"
      - "example-website": "web2.example.com:32350"
      - "example-website": "web3.example.com:31185"
...
10

Save your replica set config file.

11

Update and restart your replica set deployment.

Invoke the following Kubernetes command to update and restart your replica set:

kubectl apply -f <replica-set-conf>.yaml
12

Check the status of your deployment.

The Kubernetes Operator creates the MongoDB resources and requests the Kubernetes CA to approve the database host’s certificates. Run the following command to verify that the certificates are pending approval:

kubectl get mdb <resource-name> -o yaml -w

The status field of the output should resemble the following:

status:
  lastTransition: 2019-05-01T15:36:59Z
  message: Not all certificates have been approved by Kubernetes CA
  phase: Failed
  type: ""
  version: ""

If you do not see the status.message above, see Troubleshooting the Kubernetes Operator to help diagnose the issue.

13

Retrieve the CSRs for each host and agent in your deployment.

Invoke the following command to retrieve the CSRs for each host:

kubectl get csr

The command’s output resembles the following:

1
2
3
4
NAME                                        AGE       REQUESTOR                                                   CONDITION
my-secure-rs-0.mongodb                      33s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-rs-1.mongodb                      31s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
my-secure-rs-2.mongodb                      24s       system:serviceaccount:mongodb:mongodb-enterprise-operator   Pending
14

Approve the CSR for each host in your deployment.

Using the values returned in the NAME column, approve each certificate from the previous command’s output using the following command:

kubectl certificate approve <NAME>

Example

The following commands approve the CSRs for the replica set example:

kubectl certificate approve my-secure-rs-0.mongodb
kubectl certificate approve my-secure-rs-1.mongodb
kubectl certificate approve my-secure-rs-2.mongodb

kubectl prints a message to the console when a certificate is approved.

15

Test the connection to the replica set.

mongo --host <my-replica-set>/web1.example.com:30907,web2.example.com:32350,web3.example.com:31185 --ssl --sslAllowInvalidCertificates

Warning

Don’t use the –sslAllowInvalidCertificates flag in production. In production, share the Kubernetes CA files with client tools or applications.

If the connection succeeds, you should see:

MongoDB Enterprise <my-replica-set>:PRIMARY>