Navigation

Ops Manager Architecture in Kubernetes

You can use the Kubernetes Operator to deploy Ops Manager and MongoDB resources to a Kubernetes cluster. The Kubernetes Operator manages the lifecycle of each of these deployments differently.

The MongoDBOpsManager Custom Resource Definition

The Kubernetes Operator manages Ops Manager deployments using the MongoDBOpsManager custom resource. The Kubernetes Operator watches the custom resource’s specification for changes. When the specification changes, the Kubernetes Operator validates the changes and makes the appropriate updates to the resources in the Kubernetes cluster.

MongoDBOpsManager custom resources specification defines the following Ops Manager components:

  • the Application Database,
  • the Ops Manager application, and
  • the Backup Daemon.
Diagram showing the high-level architecture of the MongoDB Enterprise Kubernetes Operator

Application Database

For the Application Database, the Kubernetes Operator deploys a MongoDB replica set as a StatefulSet to the Kubernetes cluster. Kubernetes creates one Pod in the StatefulSet for each member that comprises your Application Database replica set. Each Pod in the StatefulSet runs a MongoDB Agent instance.

By default, each MongoDB Agent starts the bundled mongod on its Pod in the StatefulSet. If you want to use a specific MongoDB Server version for the Application Database instead, specify the version that you want to start using the spec.applicationDatabase.version setting. Each MongoDB Agent downloads the mongod version you specify from the Internet and starts it on its Pod in the StatefulSet.

After each MongoDB Agent starts mongods on its Application Database Pod, the MongoDB Agents add all mongod processes to the Application Database replica set.

You configure the number of replicas in and other configuration options for the Application Database replica set in the spec.applicationDatabase collection in the MongoDBOpsManager custom resource. The Kubernetes Operator passes this configuration to the MongoDB Agents using a secret that the Kubernetes Operator mounts to each Pod in the Application Database StatefulSet.

Each time that you update the spec.applicationDatabase collection, the Kubernetes Operator applies the changes to the MongoDB Agent configuration and the StatefulSet specification, if applicable. If the StatefulSet specification changes, Kubernetes upgrades the Pods in a rolling fashion and restarts each Pod.

The Kubernetes Operator creates a service with clusterIp=none to provide connectivity to each Application Database Pod from within the Kubernetes cluster.

If you set spec.applicationDatabase.persistent to true, the Kubernetes Operator creates a Persistent Volume Claim for each Pod in the Application Database StatefulSet.

Note

Depending on the StorageClass or the environment to which you deploy the Kubernetes Operator, Kubernetes might create the Persistent Volumes using dynamic volume provisioning.

You can customize the Persistent Volume Claims for the Application Database Pods using the spec.applicationDatabase.podSpec.persistence.single or spec.applicationDatabase.podSpec.persistence.multiple options.

Ops Manager Application

After the Application Database reaches a Running state, the Kubernetes Operator starts the Ops Manager Application. For Ops Manager, the Kubernetes Operator deploys a StatefulSet to the Kubernetes cluster. Kubernetes creates one Pod in the StatefulSet for each Ops Manager replica that you want to deploy. Each Pod contains one Ops Manager Application process.

Note

Deploy multiple Ops Manager replicas to make your deployment highly available in the event of an Ops Manager Pod failure.

The Kubernetes Operator creates a service with clusterIp=none to allow clients deployed to the Kubernetes cluster to connect to Ops Manager. To allow clients external to the Kubernetes cluster to connect to Ops Manager, configure the spec.externalConnectivity collection in the specification for your Ops Manager deployment.

Backup Daemon

If spec.backup.enabled is true, the Kubernetes Operator starts the Backup Daemon after the Ops Manager Application reaches a Running stage. For the Backup Daemon, Kubernetes Operator deploys a StatefulSet to the Kubernetes cluster. Kubernetes creates one pod in the StatefulSet for the Backup Daemon.

If you enable backup, you must provide additional fields in the spec.backup collection to configure: the oplog store and a blockstore or an S3 snapshot store.

If you enable backup, the Kubernetes Operator creates a Persistent Volume Claim for the Backup Daemon’s head database. You can configure the head database using the spec.backup.headDB setting.

The Kubernetes Operator invokes Ops Manager APIs to ensure that the Ops Manager Application’s backup configuration matches the one that you define in the custom resource definition.

Reconciling the MongoDBOpsManager Custom Resource

The following diagram describes how the Kubernetes Operator reconciles changes to the MongoDBOpsManager CustomResourceDefinition.

Diagram describing how the MongoDB Enterprise Kubernetes Operator reconciles changes to the MongoDBOpsManager Custom Resource Definition
  1. The Kubernetes Operator creates or updates the <om_resource_name>-db-config Secret. This secret contains the configurations that the MongoDB Agent uses to start the Application Database replica set.

  2. The Kubernetes Operator creates or updates the <om_resource_name>-db Application Database StatefulSet. This StatefulSet contains at least three Pods.

    • Each Pod runs one MongoDB Agent instance. Each MongoDB Agent starts a mongod instance on its pod.
    • The Kubernetes Operator mounts the <om_resource_name>-db-config Secret to each Pod. The MongoDB Agent uses this secret to configure the Application Database replica set.
  3. The Kubernetes Operator creates or updates the <om_resource_name> StatefulSet. This StatefulSet contains one Pod for each Ops Manager replica. Each Ops Manager replica connects to the Application Database.

    Note

    Most changes to the MongoDBOpsManager custom resource trigger a rolling upgrade of the Pods in the <om_resource_name> StatefulSet. Enabling TLS for the Application Database also triggers a rolling restart because the connection string to the Application Database changes.

    Changes to the following MongoDBOpsManager custom resource collections don’t trigger a rolling upgrade:

  4. The Kubernetes Operator invokes Ops Manager APIs to create an admin user. The Kubernetes Operator saves this admin user’s credentials in the <om_resource_name>-admin-key Secret. The Kubernetes Operator uses these credentials for all other Ops Manager API invocations.

    Note

    This reconciliation step happens only once: when you use the Kubernetes Operator to create an Ops Manager resource. The Kubernetes Operator skips this step when it updates the resource.

  5. The Kubernetes Operator performs a rolling upgrade of the Pods in the <om_resource_name>-db Application Database StatefulSet to enable Ops Manager to monitor it.

    Note

    This reconciliation step happens only when you enable Monitoring for an application database for the first time. This happens most often when you deploy a new Ops Manager resource.

  6. If spec.backup.enabled is true, the Kubernetes Operator creates the <om_resource_name>-backup-daemon StatefulSet or verifies that it is running. The Kubernetes Operator mounts a Persistent Volume for the head database.

    Note

    The Backup Daemon connects to the same Application Database as the Ops Manager deployment.

  7. If spec.backup.enabled is true, the Kubernetes Operator invokes Ops Manager APIs to ensure that the Ops Manager Application’s backup configuration matches the one that you define in the custom resource definition.