Navigation

Transactions

New in version 4.0.

In MongoDB, an operation on a single document is atomic. Because you can use embedded documents and arrays to capture relationships between data in a single document structure instead of normalizing across multiple documents and collections, this single-document atomicity obviates the need for multi-document transactions for many practical use cases.

However, for situations that require atomicity for updates to multiple documents or consistency between reads to multiple documents, MongoDB provides the ability to perform multi-document transactions against replica sets. Multi-document transactions can be used across multiple operations, collections, databases, and documents. Multi-document transactions provide an “all-or-nothing” proposition. When a transaction commits, all data changes made in the transaction are saved. If any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible. Until a transaction commits, no write operations in the transaction are visible outside the transaction.

Important

In most cases, multi-document transaction incurs a greater performance cost over single document writes, and the availability of multi-document transaction should not be a replacement for effective schema design. For many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions.

Transactions and Replica Sets

Multi-document transactions are available for replica sets only. Transactions for sharded clusters are scheduled for MongoDB 4.2 [1].

[1]The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.

Feature Compatibility Version (FCV)

The featureCompatibilityVersion (fCV) of all members of the replica set must be 4.0 or greater. To check the fCV for a member, connect to the member and run the following command:

db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

For more information on fCV, see setFeatureCompatibilityVersion.

Storage Engines

Multi-document transactions are available for the WiredTiger storage engine and in-memory storage engines.

If any voting member of a replica set uses the in-memory storage engine, you must set writeConcernMajorityJournalDefault to false.

Transactions and Operations

For transactions:

  • You can specify read/write (CRUD) operations on existing collections. The collections can be in different databases.
  • You cannot read/write to collections in the config, admin, or local databases.
  • You cannot write to system.* collections.
  • You cannot return the supported operation’s query plan (i.e. explain).
  • For cursors created outside of transactions, you cannot call getMore inside a transaction.
  • For cursors created in a transaction, you cannot call getMore outside the transaction.

Operations that affect the database catalog, such as creating or dropping a collection or an index, are not allowed in multi-document transactions. For example, a multi-document transaction cannot include an insert operation that would result in the creation of a new collection. See Restricted Operations.

For multi-document transactions:

Method Command Note
db.collection.aggregate() aggregate

Excluding the following stages:

db.collection.distinct() distinct  
db.collection.find() find  
  geoSearch  
delete  
findAndModify For upsert, only when run against an existing collection.
insert Only when run against an existing collection.
db.collection.save()   If an insert, only when run against an existing collection.
update For upsert, only when run against an existing collection.
  For insert operations, only when run against an existing collection.

Count Operation

To perform a count operation within a transaction, use the $count aggregation stage.

Informational Operations

Informational commands, such as isMaster, buildInfo, connectionStatus (and their helper methods) are allowed in transactions; however, they cannot be the first operation in the transaction.

Restricted Operations

The following operations are not allowed in multi-document transactions:

  • Operations that affect the database catalog, such as creating or dropping a collection or an index. For example, a multi-document transaction cannot include an insert operation that would result in the creation of a new collection.

    The listCollections and listIndexes commands and their helper methods are also excluded.

  • Non-CRUD and non-informational operations, such as createUser, getParameter, count, etc. and their helpers.

Transactions and Security

[2]If using $external authentication users (i.e. Kerberos, LDAP, x.509 users), the usernames cannot be greater than 10k bytes.

Transactions and Locks

By default, transactions waits 5 milliseconds to acquire locks required by the operations in the transaction. If the transaction cannot acquire its required locks within the 5 milliseconds, the transaction aborts.

You can use the maxTransactionLockRequestTimeoutMillis parameter to adjust how long transactions wait to acquire locks.

You can also use operation-specific timeout by setting maxTransactionLockRequestTimeoutMillis to -1.

Transactions release all locks upon abort.

Transactions and Sessions

Transactions are associated with a session. That is, you start a transaction for a session. At any given time, you can have at most one open transaction for a session.

If a session ends and it has an open transaction, the transaction aborts.

Important

When using the drivers, you must pass the session to each operation in the transaction. For more information, see Transactions and MongoDB Drivers.

Transactions and MongoDB Drivers

Clients require MongoDB drivers updated for MongoDB 4.0.

  • Java 3.8.0
  • Python 3.7.0
  • C 1.11.0
  • C# 2.7
  • Node 3.1.0
  • Ruby 2.6.0
  • Perl 2.0.0
  • PHPC 1.5.0
  • Scala 2.4.0

The following example assumes that the collections exists.

Important

To associate read and write operations with a transaction, you must pass the session to each operation in the transaction.


def update_employee_info(session):
    employees_coll = session.client.hr.employees
    events_coll = session.client.reporting.events

    with session.start_transaction(
            read_concern=ReadConcern("snapshot"),
            write_concern=WriteConcern(w="majority")):
        employees_coll.update_one(
            {"employee": 3}, {"$set": {"status": "Inactive"}},
            session=session)
        events_coll.insert_one(
            {"employee": 3, "status": {
                "new": "Inactive", "old": "Active"}},
            session=session)

        while True:
            try:
                # Commit uses write concern set at transaction start.
                session.commit_transaction()
                print("Transaction committed.")
                break
            except (ConnectionFailure, OperationFailure) as exc:
                # Can retry commit
                if exc.has_error_label(
                        "UnknownTransactionCommitResult"):
                    print("UnknownTransactionCommitResult, retrying "
                          "commit operation ...")
                    continue
                else:
                    print("Error during commit ...")
                    raise
try (ClientSession clientSession = client.startSession()) {
    clientSession.startTransaction();

    employeesCollection.updateOne(clientSession,
            Filters.eq("employee", 3),
            Updates.set("status", "Inactive"));
    eventsCollection.insertOne(clientSession,
            new Document("employee", 3).append("status", new Document("new", "Inactive").append("old", "Active")));

    commitWithRetry(clientSession);
}

Note that the session is passed to each operation in the transaction.

Transactions and the mongo Shell

The following mongo shell methods are available for transactions:

Transactions and Retryable Writes

Highly Available Applications

Regardless of the database system, whether MongoDB or relational databases, applications should take measures to handle errors during transaction commits and incorporate retry logic for transactions.

Retry Transaction

The individual write operations inside the transaction are not retryable, regardless of whether retryWrites is set to true.

If an operation encounters an error, the returned error may have an errorLabels array field. If the error is a transient error, the errorLabels array field contains "TransientTransactionError" as an element and the transaction as a whole can be retried.

For example, the following helper runs a function and retries the function if a "TransientTransactionError" is encountered:

// Runs the txnFunc and retries if TransientTransactionError encountered

function runTransactionWithRetry(txnFunc, session) {
    while (true) {
        try {
            txnFunc(session);  // performs transaction
            break;
        } catch (error) {
            print("Transaction aborted. Caught exception during transaction.");

            // If transient error, retry the whole transaction
            if ( error.hasOwnProperty("errorLabels") && error.errorLabels.includes( "TransientTransactionError")  ) {
                print("TransientTransactionError, retrying transaction ...");
                continue;
            } else {
                throw error;
            }
        }
    }
}
def run_transaction_with_retry(txn_func, session):
    while True:
        try:
            txn_func(session)  # performs transaction
            break
        except (ConnectionFailure, OperationFailure) as exc:
            print("Transaction aborted. Caught exception during "
                  "transaction.")

            # If transient error, retry the whole transaction
            if exc.has_error_label("TransientTransactionError"):
                print("TransientTransactionError, retrying"
                      "transaction ...")
                continue
            else:
                raise
void runTransactionWithRetry(Runnable transactional) {
    while (true) {
        try {
            transactional.run();
            break;
        } catch (MongoException e) {
            System.out.println("Transaction aborted. Caught exception during transaction.");

            if (e.hasErrorLabel(MongoException.TRANSIENT_TRANSACTION_ERROR_LABEL)) {
                System.out.println("TransientTransactionError, aborting transaction and retrying ...");
                continue;
            } else {
                throw e;
            }
        }
    }
}

Retry Commit Operation

The commit operations are retryable write operations. If the commit operation operation encounters an error, MongoDB drivers retry the operation a single time regardless of whether retryWrites is set to true.

If the commit operation encounters an error, MongoDB returns an error with an errorLabels array field. If the error is a transient commit error, the errorLabels array field contains "UnknownTransactionCommitResult" as an element and the commit operation can be retried.

In addition to the single retry behavior provided by the MongoDB drivers, applications should take measures to handle "UnknownTransactionCommitResult" errors during transaction commits.

For example, the following helper commits a transaction and retries if a "UnknownTransactionCommitResult" is encountered:

// Retries commit if UnknownTransactionCommitResult encountered

function commitWithRetry(session) {
    while (true) {
        try {
            session.commitTransaction(); // Uses write concern set at transaction start.
            print("Transaction committed.");
            break;
        } catch (error) {
            // Can retry commit
            if (error.hasOwnProperty("errorLabels") && error.errorLabels.includes( "UnknownTransactionCommitResult") ) {
                print("UnknownTransactionCommitResult, retrying commit operation ...");
                continue;
            } else {
                print("Error during commit ...");
                throw error;
            }
       }
    }
}
def commit_with_retry(session):
    while True:
        try:
            # Commit uses write concern set at transaction start.
            session.commit_transaction()
            print("Transaction committed.")
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # Can retry commit
            if exc.has_error_label("UnknownTransactionCommitResult"):
                print("UnknownTransactionCommitResult, retrying "
                      "commit operation ...")
                continue
            else:
                print("Error during commit ...")
                raise
void commitWithRetry(ClientSession clientSession) {
    while (true) {
        try {
            clientSession.commitTransaction();
            System.out.println("Transaction committed");
            break;
        } catch (MongoException e) {
            // can retry commit
            if (e.hasErrorLabel(MongoException.UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL)) {
                System.out.println("UnknownTransactionCommitResult, retrying commit operation ...");
                continue;
            } else {
                System.out.println("Exception during commit ...");
                throw e;
            }
        }
    }
}

Retry Transaction and Commit Operation

Incorporating logic to retrying the transaction for transient errors and retrying the commit, the full code example is:

// Runs the txnFunc and retries if TransientTransactionError encountered

function runTransactionWithRetry(txnFunc, session) {
    while (true) {
        try {
            txnFunc(session);  // performs transaction
            break;
        } catch (error) {
            // If transient error, retry the whole transaction
            if ( error.hasOwnProperty("errorLabels") && error.errorLabels.includes("TransientTransactionError")  ) {
                print("TransientTransactionError, retrying transaction ...");
                continue;
            } else {
                throw error;
            }
        }
    }
}

// Retries commit if UnknownTransactionCommitResult encountered

function commitWithRetry(session) {
    while (true) {
        try {
            session.commitTransaction(); // Uses write concern set at transaction start.
            print("Transaction committed.");
            break;
        } catch (error) {
            // Can retry commit
            if (error.hasOwnProperty("errorLabels") && error.errorLabels.includes("UnknownTransactionCommitResult") ) {
                print("UnknownTransactionCommitResult, retrying commit operation ...");
                continue;
            } else {
                print("Error during commit ...");
                throw error;
            }
       }
    }
}

// Updates two collections in a transactions

function updateEmployeeInfo(session) {
    employeesCollection = session.getDatabase("hr").employees;
    eventsCollection = session.getDatabase("reporting").events;

    session.startTransaction( { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } } );

    try{
        employeesCollection.updateOne( { employee: 3 }, { $set: { status: "Inactive" } } );
        eventsCollection.insertOne( { employee: 3, status: { new: "Inactive", old: "Active" } } );
    } catch (error) {
        print("Caught exception during transaction, aborting.");
        session.abortTransaction();
        throw error;
    }

    commitWithRetry(session);
}

// Start a session.
session = db.getMongo().startSession( { mode: "primary" } );

try{
   runTransactionWithRetry(updateEmployeeInfo, session);
} catch (error) {
   // Do something with error
} finally {
   session.endSession();
}

def run_transaction_with_retry(txn_func, session):
    while True:
        try:
            txn_func(session)  # performs transaction
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # If transient error, retry the whole transaction
            if exc.has_error_label("TransientTransactionError"):
                print("TransientTransactionError, retrying "
                      "transaction ...")
                continue
            else:
                raise

def commit_with_retry(session):
    while True:
        try:
            # Commit uses write concern set at transaction start.
            session.commit_transaction()
            print("Transaction committed.")
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # Can retry commit
            if exc.has_error_label("UnknownTransactionCommitResult"):
                print("UnknownTransactionCommitResult, retrying "
                      "commit operation ...")
                continue
            else:
                print("Error during commit ...")
                raise

# Updates two collections in a transactions

def update_employee_info(session):
    employees_coll = session.client.hr.employees
    events_coll = session.client.reporting.events

    with session.start_transaction(
            read_concern=ReadConcern("snapshot"),
            write_concern=WriteConcern(w="majority")):
        employees_coll.update_one(
            {"employee": 3}, {"$set": {"status": "Inactive"}},
            session=session)
        events_coll.insert_one(
            {"employee": 3, "status": {
                "new": "Inactive", "old": "Active"}},
            session=session)

        commit_with_retry(session)

# Start a session.
with client.start_session() as session:
    try:
        run_transaction_with_retry(update_employee_info, session)
    except Exception as exc:
        # Do something with error.
        raise

void runTransactionWithRetry(Runnable transactional) {
    while (true) {
        try {
            transactional.run();
            break;
        } catch (MongoException e) {
            System.out.println("Transaction aborted. Caught exception during transaction.");

            if (e.hasErrorLabel(MongoException.TRANSIENT_TRANSACTION_ERROR_LABEL)) {
                System.out.println("TransientTransactionError, aborting transaction and retrying ...");
                continue;
            } else {
                throw e;
            }
        }
    }
}

void commitWithRetry(ClientSession clientSession) {
    while (true) {
        try {
            clientSession.commitTransaction();
            System.out.println("Transaction committed");
            break;
        } catch (MongoException e) {
            // can retry commit
            if (e.hasErrorLabel(MongoException.UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL)) {
                System.out.println("UnknownTransactionCommitResult, retrying commit operation ...");
                continue;
            } else {
                System.out.println("Exception during commit ...");
                throw e;
            }
        }
    }
}

void updateEmployeeInfo() {
    MongoCollection<Document> employeesCollection = client.getDatabase("hr").getCollection("employees");
    MongoCollection<Document> eventsCollection = client.getDatabase("hr").getCollection("events");

    try (ClientSession clientSession = client.startSession()) {
        clientSession.startTransaction();

        employeesCollection.updateOne(clientSession,
                Filters.eq("employee", 3),
                Updates.set("status", "Inactive"));
        eventsCollection.insertOne(clientSession,
                new Document("employee", 3).append("status", new Document("new", "Inactive").append("old", "Active")));

        commitWithRetry(clientSession);
    }
}


void updateEmployeeInfoWithRetry() {
    runTransactionWithRetry(this::updateEmployeeInfo);
}

Atomicity

Multi-document transactions are atomic:

  • When a transaction commits, all data changes made in the transaction are saved and visible outside the transaction. Until a transaction commits, the data changes made in the transaction are not visible outside the transaction.
  • When a transaction aborts, all data changes made in the transaction are discarded without ever becoming visible. For example, if any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible.

Read Preference

Multi-document transactions that contain read operations must use read preference primary.

All operations in a given transaction must route to the same member.

Transaction Options

session.startTransaction( {
   readConcern: { level: <level> },
   writeConcern: { w: <value>, j: <boolean>, wtimeout: <number> }
} );

Read Concern

Multi-document transactions support read concern "snapshot", "local", and "majority":

  • For "local" and "majority" read concern, MongoDB may sometimes substitute a stronger read concern.
  • For "majority" read concern, if the transaction commits with write concern “majority”, transaction operations are guaranteed to have read majority-committed data. Otherwise, the "majority" read concern provides no guarantees that read operations read majority-committed data.
  • For "snapshot" read concern, if the transaction commits with write concern “majority”, the transaction operations are guaranteed to have read from a snapshot of majority committed data. Otherwise, the "snapshot" read concern provides no guarantee that read operations used a snapshot of majority-committed data.

You set the read concern at the transaction level, not at the individual operation level. The operations in the transaction will use the transaction-level read concern. Any read concern set at the collection and database level is ignored inside the transaction. If the transaction-level read concern is explicitly specified, the client level read concern is also ignored inside the transaction.

You can set the transaction read concern at the transaction start.

If unspecified at the transaction start, transactions use the session-level read concern or, if that is unset, the client-level read concern.

Write Concern

You set the write concern at the transaction level, not at the individual operation level. At the time of the commit, transactions use the transaction level write concern to commit the write operations. Individual operations inside the transaction ignore write concerns. Do not explicitly set the write concern for the individual write operations inside transactions.

You can set the write concern for the transaction commit at the transaction start.

  • If unspecified at the transaction start, transactions use the session-level write concern for the commit or, if that is unset, the client-level write concern.
  • Write concern w: 0 is not supported for transactions.
  • If you commit using w: 1 write concern, your transaction can be rolled back if there is a failover.
  • If the transaction commits with write concern “majority” and has specified read concern "snapshot" read concern, transaction operations are guaranteed to have read from a snapshot of majority-committed data. Otherwise, the "snapshot" read concern provides no guarantees that read operations used a snapshot of majority-committed data.
  • If the transaction commits with write concern “majority” and has specified read concern "majority" read concern, transaction operations are guaranteed to have read majority-committed data. Otherwise, the "majority" read concern provides no guarantees that read operations read majority-committed data.