Migrate from Compose to MongoDB Atlas

Overview

This guide walks you through how to migrate data from Compose to MongoDB Atlas using the Atlas Live Migration process.

The MongoDB Atlas Live Migration Service helps you migrate MongoDB databases to our fully managed cloud database, MongoDB Atlas, quickly and securely. It works by connecting to your existing MongoDB database and synchronizing it with a cluster running in Atlas all while your application continues to function normally. Once the data between the two clusters has been synchronized, you can simply update the database connection string in your application to cut over to your cluster in Atlas.

Time required: 20 minutes

What You'll Need

Important

If you have SSL enabled on your Compose deployment, you will need access to your SSL certificate to complete the Live Migration process.

In some Compose deployments, you can no longer view your SSL certificate in the Compose UI. If this is the case for your deployment, you have two options:

Contact Compose directly to request your SSL certificate.
Use MongoMirror to migrate your data to Atlas.

See the Compose documentation for more information about SSL certificates.

Your data is currently in a MongoDB database.
This guide focuses on migrating to Atlas from an existing MongoDB deployment on Compose.
Your current MongoDB database is running MongoDB 2.6 or higher.
Atlas supports the latest versions of MongoDB: 4.2, 4.4, 5.0, and 6.0. If you're running MongoDB version 2.6 or greater, the Atlas Live Migration Service can move your data directly into a newer database version. Update your MongoDB drivers and make any necessary code changes at the application level to ensure compatibility. If you're running a version older than 2.6, see Upgrade MongoDB to 2.6 for upgrade instructions.
Your current deployment is a MongoDB replica set or sharded cluster.
If your deployment is currently a standalone instance, you must first convert it to a replica set.
Live migration of data from sharded clusters is not supported. Your destination cluster may be sharded, but your source cluster must be an unsharded replica set.
(Optional) Enabled authentication on your source deployment.
The migration process requires that authentication is enabled on your source cluster in AWS. See Enable Auth for instructions on enabling authentication. You can verify that authentication is enabled on your source cluster using the mongosh command:
```
mongosh <mongodb-connection-string> -u <mongodb-username> -p --authenticationDatabase admin
```

The database user from your source cluster that you will use to perform the migration has the required MongoDB roles.

The readAnyDatabase role.
The clusterMonitor role.
The backup role.

To verify that the database user that will run the Live Migration process has these roles, run the db.getUser() command on the admin database.

use admin
db.getUser("admin")
{
  "_id" : "admin.admin",
  "user" : "admin",
  "db" : "admin",
  "roles" : [
    {
      "role" : "backup",
      "db" : "admin"
    },
    {
      "role" : "clusterMonitor",
      "db" : "admin"
    }
    {
      "role" : "readAnyDatabase",
      "db" : "admin"
    }
  ]
} ...

In addition, the database user from your source cluster in Compose must have the role to read the oplog on your admin database. See Oplog Access. You obtain access to this role when you add the oplog user in Compose in the following procedure. If you can't grant all of these permissions to the database user from your source cluster in Compose, the Live Migration process will not work. In this case, use mongodump and mongorestore to migrate your data to Atlas.

Procedure

Create an Atlas deployment.

If you don't already have an Atlas deployment, create one now. You'll need a cluster tier of M10 or larger to perform Live Migration.

Log in to your Compose account.

Note

It will be helpful during the migration process to keep one browser window open on your Compose deployment console and one window open on your Atlas console.

Create an oplog user.

Click the Add-ons link in the left-side navigation. If you don't have the Oplog Access add-on, add it with the Add button.

If you already have the Oplog Access add-on, click Configure to see the oplog user username and password.

Shows the oplog username and password fields within the
Oplog Access add-on.

click to enlarge

To perform the migration process, the oplog user must have the following privileges:

The readAnyDatabase role.
The clusterMonitor role.
The backup role.

If you can't grant these permissions to the oplog user, the Live Migration process will not work. In this case, use mongodump and mongorestore to migrate your data to Atlas.

Review migration steps.

Read through the overview of migration steps in the Live Migration dialog window, then click the green I'm ready to migrate button.

Add IP address ranges to your Compose deployment whitelist.

Open two browser tabs:

For the Atlas Live Migration process dialog from the previous step.
For your Compose deployment dashboard.

On your Compose deployment dashboard, click the Security link in the left-side navigation. The Whitelist TCP/HTTP IPs section displays a list of IP address ranges which are allowed to access your Compose deployment.

Add the IP address range which is listed at the top of the Atlas Migration process dialog window.

Shows IP address range entered in the Whiltelist TCP/HTTP
IPs section.

click to enlarge

Note

Your Atlas migration IP address ranges may be different from those shown here.

Add the hostname and port of your Compose deployment to the Atlas Live Migration dialog.

On the Oplog Access add-on page, you'll find a connection string with a hostname and port for oplog access. Copy them to the Atlas Live Migration dialog.

Shows field filled in with a hostname and port.

Enter the oplog user's credentials in the Live Migration dialog.

Enter the username and password for oploguser in the Atlas Live Migration dialog window.

Enter your Compose TLS/SSL Certificate data.

If you don't have TLS/SSL enabled on your Compose deployment, skip this step.

Important

In some Compose deployments, you can no longer view your TLS/SSL certificate in the Compose UI. If this is the case for your deployment, you have two options:

Contact Compose directly to request your TLS/SSL certificate.
Use MongoMirror to migrate your data to Atlas.

See the Compose documentation for more information about SSL certificates.

On the Oplog Access add-on page, you'll find an SSL certificate. Copy it to the CAFile text box on the Atlas Live Migration dialog.

Shows an SSL certificate pasted into the CAFile text box.

Note

Copy the entire certificate file, including the BEGIN CERTIFICATE and END CERTIFICATE lines.

Validate your Live Migration form.

Click the Validate button to check that all your form fields are valid and your clusters are ready for migration. When your form is validated, you're ready to begin the migration.

Click Start Migration.

A countdown timer in a progress bar indicates how much time remains before your target cluster is ready to migrate data from your source cluster. Wait until the countdown timer and the Prepare to Cutover button are green before proceeding to the next step.

Click Prepare to Cutover.

Perform the cutover.

When Atlas detects that the source and destination clusters are nearly in sync, it starts an extendable 72 hour timer to begin the cutover procedure. If the 72 hour period passes, Atlas stops synchronizing with the source cluster. You can extend the time remaining by 24 hours by clicking the Extend time hyperlink below the <time> left to cut over timer.

Once you are prepared to cut your applications over to the destination Atlas cluster, click Prepare to Cutover.

Atlas displays a walk-through screen with instructions on how to proceed with the cutover. These steps are also outlined below:

Stop your application. This ensures that no additional writes are generated to the source cluster.
Wait for the optime gap to reach zero. When the counter reaches zero, the source and destination clusters are in sync.
Restart your application using the new connection string provided in step 3 of the Live Migrate cutover UI.

Once you have completed the cutover procedure and confirmed your applications are working normally with the Atlas cluster, click Cut Over to complete the migration procedure. This allows Atlas to:

Mark the migration plan as complete.
Remove the Application Server subnets from the destination cluster IP access list.
Remove the MongoDB user that Live Migrate used to import data to the destination cluster.

Migration Support

If you have any questions regarding migration beyond what is covered in this documentation, or if you encounter an error during migration, please see the Atlas documentation on requesting support.

Summary

You created an Atlas cluster, migrated data from your old Compose cluster, and updated your applications to use the new Atlas Cluster. Congratulations and welcome to MongoDB Atlas!