Navigation

Transactions

In MongoDB, an operation on a single document is atomic. Because you can use embedded documents and arrays to capture relationships between data in a single document structure instead of normalizing across multiple documents and collections, this single-document atomicity obviates the need for multi-document transactions for many practical use cases.

For situations that require atomicity of reads and writes to multiple documents (in a single or multiple collections), MongoDB supports multi-document transactions. With distributed transactions, transactions can be used across multiple operations, collections, databases, documents, and shards.

Transactions API

The following example highlights the key components of the transactions API:

The example uses the new callback API for working with transactions, which starts a transaction, executes the specified operations, and commits (or aborts on error). The new callback API also incorporates retry logic for TransientTransactionError or UnknownTransactionCommitResult commit errors.

Important

  • For transactions on MongoDB 4.2 deployments (replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.
  • When using the drivers, each operation in the transaction must be associated with the session (i.e. pass in the session to each operation).
# For a replica set, include the replica set name and a seedlist of the members in the URI string; e.g.
# uriString = 'mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017/admin?replicaSet=myRepl'
# For a sharded cluster, connect to the mongos instances; e.g.
# uriString = 'mongodb://mongos0.example.com:27017,mongos1.example.com:27017/admin'

client = MongoClient(uriString)

my_write_concern_majority = WriteConcern('majority', wtimeout=1000)

# Prereq: Create collections. CRUD operations in transactions must be on existing collections.

client.get_database(
    'mydb1',
    write_concern=my_write_concern_majority).foo.insert_one({'abc': 0})
client.get_database(
    'mydb2',
    write_concern=my_write_concern_majority).bar.insert_one({'xyz': 0})

# Step 1: Define the callback that specifies the sequence of operations to perform inside the transactions.

def callback(my_session):
    collection_one = my_session.client.mydb1.foo
    collection_two = my_session.client.mydb2.bar

    # Important:: You must pass the session to the operations.

    collection_one.insert_one({'abc': 1}, session=my_session)
    collection_two.insert_one({'xyz': 999}, session=my_session)

#. Step 2: Start a client session.

with client.start_session() as session:

    # Step 3: Use with_transaction to start a transaction, execute the callback, and commit (or abort on error).

    session.with_transaction(callback,
                             read_concern=ReadConcern('local'),
                             write_concern=my_write_concern_majority,
                             read_preference=ReadPreference.PRIMARY)

The example uses the new callback API for working with transactions, which starts a transaction, executes the specified operations, and commits (or aborts on error). The new callback API also incorporates retry logic for TransientTransactionError or UnknownTransactionCommitResult commit errors.

Important

  • For transactions on MongoDB 4.2 deployments (replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.
  • When using the drivers, each operation in the transaction must be associated with the session (i.e. pass in the session to each operation).
/*
  For a replica set, include the replica set name and a seedlist of the members in the URI string; e.g.
  String uri = "mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017/admin?replicaSet=myRepl";
  For a sharded cluster, connect to the mongos instances; e.g.
  String uri = "mongodb://mongos0.example.com:27017,mongos1.example.com:27017:27017/admin";
 */

final MongoClient client = MongoClients.create(uri);

/*
    Prereq: Create collections. CRUD operations in transactions must be on existing collections.
 */

client.getDatabase("mydb1").getCollection("foo")
        .withWriteConcern(WriteConcern.MAJORITY).insertOne(new Document("abc", 0));
client.getDatabase("mydb2").getCollection("bar")
        .withWriteConcern(WriteConcern.MAJORITY).insertOne(new Document("xyz", 0));

/* Step 1: Start a client session. */

final ClientSession clientSession = client.startSession();

/* Step 2: Optional. Define options to use for the transaction. */

TransactionOptions txnOptions = TransactionOptions.builder()
        .readPreference(ReadPreference.primary())
        .readConcern(ReadConcern.LOCAL)
        .writeConcern(WriteConcern.MAJORITY)
        .build();

/* Step 3: Define the sequence of operations to perform inside the transactions. */

TransactionBody txnBody = new TransactionBody<String>() {
    public String execute() {
        MongoCollection<Document> coll1 = client.getDatabase("mydb1").getCollection("foo");
        MongoCollection<Document> coll2 = client.getDatabase("mydb2").getCollection("bar");

        /*
           Important:: You must pass the session to the operations.
         */
        coll1.insertOne(clientSession, new Document("abc", 1));
        coll2.insertOne(clientSession, new Document("xyz", 999));
        return "Inserted into collections in different databases";
    }
};
try {
    /*
       Step 4: Use .withTransaction() to start a transaction,
       execute the callback, and commit (or abort on error).
    */

    clientSession.withTransaction(txnBody, txnOptions);
} catch (RuntimeException e) {
    // some error handling
} finally {
    clientSession.close();
}

The example uses the new callback API for working with transactions, which starts a transaction, executes the specified operations, and commits (or aborts on error). The new callback API also incorporates retry logic for TransientTransactionError or UnknownTransactionCommitResult commit errors.

Important

  • For transactions on MongoDB 4.2 deployments (replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.
  • When using the drivers, each operation in the transaction must be associated with the session (i.e. pass in the session to each operation).
/*
  For a replica set, include the replica set name and a seedlist of the members in the URI string; e.g.
  uri = 'mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017/admin?replicaSet=myRepl';
  For a sharded cluster, connect to the mongos instances; e.g.
  uri = 'mongodb://mongos0.example.com:27017,mongos1.example.com:27017:27017/admin';
 */

const client = new MongoClient(uri);
await client.connect();

/*
  Prerequisites: Assumes the collections already exist.
  CRUD operations in transactions must be on existing collections.
 */

/* Step 1: Start a client session. The default transaction options are optional.*/
const session = client.startSession({defaultTransactionOptions: {
   readConcern: { level: 'local' },
   writeConcern: { w: 'majority' },
   readPreference: 'primary'
}});


try {
  /*
     Step 2: Use withTransaction method to start a transaction,
     execute the callback, and commit (or abort on error).
     The withTransaction method only supports functions that return a `Promise`.
   */
  await session.withTransaction(async function() {
     const coll1 = client.db('mydb1').collection('foo');
     const coll2 = client.db('mydb2').collection('bar');

     // Important:: You must pass the session to the operations.
     await coll1.insertOne({ abc: 1 }, { session });

     // Important:: You must pass the session to the operations.
     await coll2.insertOne({ xyz: 999 }, { session });
  });
} catch (err) {
  // transaction failed after attempted retry
  // some error handling
}

The following example omits retry logic and robust error handling for simplicity’s sake.

Important

  • For transactions on MongoDB 4.2 deployments (replica sets and sharded clusters)*, clients must use MongoDB drivers updated for MongoDB 4.2.
  • When using the driver, each operation in the transaction must be associated with the session (i.e. pass in the session to each operation).
 /*  Prereq: Create collections. CRUD operations in transactions must be on existing collections. */
 $client->mydb1->createCollection('foo', ['writeConcern' => new \MongoDB\Driver\WriteConcern('majority')]);
 $client->mydb2->createCollection('bar', ['writeConcern' => new \MongoDB\Driver\WriteConcern('majority')]);

 /* Step 1: Start a client session. */

 $session = $client->startSession();

 /* Step 2: Start a transaction.  The transaction options are optional.*/
 $session->startTransaction([
     'readConcern' => new \MongoDB\Driver\ReadConcern("local"),
     'writeConcern' => new \MongoDB\Driver\WriteConcern(\MongoDB\Driver\WriteConcern::MAJORITY)
 ]);

 /*
    Step 3: Specify the operations to perform inside the transaction.
    Important:: You must pass the session to the operations.
  */

 $client->mydb1->foo->insertOne(
     ['abc' => 1],
     ['session' => $session]
 );
 $client->mydb2->bar->insertOne(
     ['xyz' => 999],
     ['session' => $session]
 );

/* Step 4: Commit the Transaction. */

 $session->commitTransaction()

Note

The mongo shell example omits retry logic and robust error handling for simplicity’s sake. For a more practical example of incorporating transactions in applications, see Transaction Error Handling instead.

// Prereq: Create collections. CRUD operations in transactions must be on existing collections.
db.getSiblingDB("mydb1").foo.insert( {abc: 0}, { writeConcern: { w: "majority", wtimeout: 2000 } } );
db.getSiblingDB("mydb2").bar.insert( {xyz: 0}, { writeConcern: { w: "majority", wtimeout: 2000 } } );

// Start a session.
session = db.getMongo().startSession( { readPreference: { mode: "primary" } } );

coll1 = session.getDatabase("mydb1").foo;
coll2 = session.getDatabase("mydb2").bar;

// Start a transaction
session.startTransaction( { readConcern: { level: "local" }, writeConcern: { w: "majority" } } );

// Operations inside the transaction
try {
   coll1.insertOne( { abc: 1 } );
   coll2.insertOne( { xyz: 999 } );
} catch (error) {
   // Abort transaction on error
   session.abortTransaction();
   throw error;
}

// Commit the transaction using write concern set at transaction start
session.commitTransaction();

session.endSession();

Transactions and Atomicity

Distributed Transactions and Multi-Document Transactions

Starting in MongoDB 4.2, the two terms are synonymous. Distributed transactions refer to multi-document transactions on sharded clusters and replica sets. Multi-document transactions (whether on sharded clusters or replica sets) are also known as distributed transactions starting in MongoDB 4.2.

For situations that require atomicity of reads and writes to multiple documents (in a single or multiple collections), MongoDB supports multi-document transactions:

  • In version 4.0, MongoDB supports multi-document transactions on replica sets.

  • In version 4.2, MongoDB introduces distributed transactions, which adds support for multi-document transactions on sharded clusters and incorporates the existing support for multi-document transactions on replica sets.

    To use transactions on MongoDB 4.2 deployments(replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.

Multi-document transactions are atomic (i.e. provide an “all-or-nothing” proposition):

  • When a transaction commits, all data changes made in the transaction are saved and visible outside the transaction. That is, a transaction will not commit some of its changes while rolling back others.

    Until a transaction commits, the data changes made in the transaction are not visible outside the transaction.

    However, when a transaction writes to multiple shards, not all outside read operations need to wait for the result of the committed transaction to be visible across the shards. For example, if a transaction is committed and write 1 is visible on shard A but write 2 is not yet visible on shard B, an outside read at read concern "local" can read the results of write 1 without seeing write 2.

  • When a transaction aborts, all data changes made in the transaction are discarded without ever becoming visible. For example, if any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible.

Important

In most cases, multi-document transaction incurs a greater performance cost over single document writes, and the availability of multi-document transactions should not be a replacement for effective schema design. For many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions.

For additional transactions usage considerations (such as runtime limit and oplog size limit), see also Production Considerations.

Transactions and Operations

Distributed transactions can be used across multiple operations, collections, databases, documents, and, starting in MongoDB 4.2, shards.

For transactions:

  • You can specify read/write (CRUD) operations on existing collections. The collections can be in different databases. For a list of CRUD operations, see CRUD Operations.
  • You cannot write to capped collections. (Starting in MongoDB 4.2)
  • You cannot read/write to collections in the config, admin, or local databases.
  • You cannot write to system.* collections.
  • You cannot return the supported operation’s query plan (i.e. explain).
  • For cursors created outside of a transaction, you cannot call getMore inside the transaction.
  • For cursors created in a transaction, you cannot call getMore outside the transaction.
  • Starting in MongoDB 4.2, you cannot specify killCursors as the first operation in a transaction.

Operations that affect the database catalog, such as creating or dropping a collection or an index, are not allowed in transactions. For example, a transaction cannot include an insert operation that would result in the creation of a new collection. See Restricted Operations.

Tip

When creating or dropping a collection immediately before starting a transaction, if the collection is accessed within the transaction, issue the create or drop operation with write concern "majority" to ensure that the transaction can acquire the required locks.

Count Operation

To perform a count operation within a transaction, use the $count aggregation stage or the $group (with a $sum expression) aggregation stage.

MongoDB drivers compatible with the 4.0 features provide a collection-level API countDocuments(filter, options) as a helper method that uses the $group with a $sum expression to perform a count. The 4.0 drivers have deprecated the count() API.

Starting in MongoDB 4.0.3, the mongo shell provides the db.collection.countDocuments() helper method that uses the $group with a $sum expression to perform a count.

Distinct Operation

To perform a distinct operation within a transaction:

  • For unsharded collections, you can use the db.collection.distinct() method/the distinct command as well as the aggregation pipeline with the $group stage.

  • For sharded collections, you cannot use the db.collection.distinct() method or the distinct command.

    To find the distinct values for a sharded collection, use the aggregation pipeline with the $group stage instead. For example:

    • Instead of db.coll.distinct("x"), use

      db.coll.aggregate([
         { $group: { _id: null, distinctValues: { $addToSet: "$x" } } },
         { $project: { _id: 0 } }
      ])
      
    • Instead of db.coll.distinct("x", { status: "A" }), use:

      db.coll.aggregate([
         { $match: { status: "A" } },
         { $group: { _id: null, distinctValues: { $addToSet: "$x" } } },
         { $project: { _id: 0 } }
      ])
      

    The pipeline returns a cursor to a document:

    { "distinctValues" : [ 2, 3, 1 ] }
    

    Iterate the cursor to access the results document.

Informational Operations

Informational commands, such as isMaster, buildInfo, connectionStatus (and their helper methods) are allowed in transactions; however, they cannot be the first operation in the transaction.

Restricted Operations

The following operations are not allowed in transactions:

  • Operations that affect the database catalog, such as creating or dropping a collection or an index. For example, a transaction cannot include an insert operation that would result in the creation of a new collection.

    The listCollections and listIndexes commands and their helper methods are also excluded.

  • Non-CRUD and non-informational operations, such as createUser, getParameter, count, etc. and their helpers.

Transactions and Sessions

  • Transactions are associated with a session; i.e. you start a transaction for a session.
  • At any given time, you can have at most one open transaction for a session.
  • When using the drivers, each operation in the transaction must be associated with the session. Refer to your driver specific documentation for details.
  • If a session ends and it has an open transaction, the transaction aborts.

Read Concern/Write Concern/Read Preference

Transactions and Read Preference

Operations in a transaction use the transaction-level read preference.

Using the drivers, you can set the transaction-level read preference at the transaction start:

  • If the transaction-level read preference is unset, the transaction uses the session-level read preference.
  • If transaction-level and the session-level read preference are unset, the transaction uses the client-level read preference. By default, the client-level read preference is primary.

Multi-document transactions that contain read operations must use read preference primary. All operations in a given transaction must route to the same member.

Transactions and Read Concern

Operations in a transaction use the transaction-level read concern. That is, any read concern set at the collection and database level is ignored inside the transaction.

You can set the transaction-level read concern at the transaction start.

  • If the transaction-level read concern is unset, the transaction-level read concern defaults to the session-level read concern.
  • If transaction-level and the session-level read concern are unset, the transaction-level read concern defaults to the client-level read concern. By default, client-level read concern is "local" for reads against the primary. See also Transactions and Read Preference.

Transactions support the following read concern levels:

"local"

  • Read concern "local" returns the most recent data available from the node but can be rolled back.
  • For transactions on sharded cluster, "local" read concern cannot guarantee that the data is from the same snapshot view across the shards. If snapshot isolation is required, use "snapshot" read concern.

"majority"

  • Read concern "majority" returns data that has been acknowledged by a majority of the replica set members (i.e. data cannot be rolled back) if the transaction commits with write concern “majority”.
  • If the transaction does not use write concern “majority” for the commit, the "majority" read concern provides no guarantees that read operations read majority-committed data.
  • For transactions on sharded cluster, "majority" read concern cannot guarantee that the data is from the same snapshot view across the shards. If snapshot isolation is required, use "snapshot" read concern.

"snapshot"

  • Read concern "snapshot" returns data from a snapshot of majority committed data if the transaction commits with write concern “majority”.
  • If the transaction does not use write concern “majority” for the commit, the "snapshot" read concern provides no guarantee that read operations used a snapshot of majority-committed data.
  • For transactions on sharded clusters, the "snapshot" view of the data is synchronized across shards.

Transactions and Write Concern

Transactions use the transaction-level write concern to commit the write operations. Write operations inside transactions must be issued without explicit write concern specification and use the default write concern. At commit time, the writes are then commited using the transaction-level write concern.

Tip

Do not explicitly set the write concern for the individual write operations inside a transaction. Setting write concerns for the individual write operations inside a transaction results in an error.

You can set the transaction-level write concern at the transaction start:

  • If the transaction-level write concern is unset, the transaction-level write concern defaults to the session-level write concern for the commit.
  • If the transaction-level write concern and the session-level write concern are unset, transaction-level write concern defaults to the client-level write concern. By default, client-level write concern is w: 1.

Transactions support all write concern w values, including:

w: 1

  • Write concern w: 1 returns acknowledgement after the commit has been applied to the primary.

    Important

    When you commit with w: 1, your transaction can be rolled back if there is a failover.

  • When you commit with w: 1 write concern, transaction-level "majority" read concern provides no guarantees that read operations in the transaction read majority-committed data.

  • When you commit with w: 1 write concern, transaction-level "snapshot" read concern provides no guarantee that read operations in the transaction used a snapshot of majority-committed data.

w: "majority"

  • Write concern w: "majority" returns acknowledgement after the commit has been applied to a majority (M) of voting members; i.e. the commit has been applied to the primary and (M-1) voting secondaries.
  • When you commit with w: "majority" write concern, transaction-level "majority" read concern guarantees that operations have read majority-committed data. For transactions on sharded clusters, this view of the majority-committed data is not synchronized across shards.
  • When you commit with w: "majority" write concern, transaction-level "snapshot" read concern guarantees that operations have from a synchronized snapshot of majority-committed data.

General Information

Production Considerations

For various production considerations with using transactions, see Production Considerations. In addition, or sharded clusters, see also Production Considerations (Sharded Clusters).

Arbiters

Transactions whose write operations span multiple shards will error and abort if any transaction operation reads from or writes to a shard that contains an arbiter.

See also Disabled Read Concern Majority for transaction restrictions on shards that have disabled read concern majority.

Disabled Read Concern Majority

A 3-member PSA (Primary-Secondary-Arbiter) replica set or a sharded cluster with 3-member PSA shards may have disabled read concern majority (--enableMajorityReadConcern false or replication.enableMajorityReadConcern: false)

On sharded clusters,
  • If a transaction involves a shard that has disabled read concern “majority”, you cannot use read concern "snapshot" for the transaction. You can only use read concern "local" or "majority" for the transaction. If you use read concern "snapshot", the transaction errors and aborts.

    readConcern level 'snapshot' is not supported in sharded clusters when enableMajorityReadConcern=false.
    
  • Transactions whose write operations span multiple shards will error and abort if any of the transaction’s read or write operations involves a shard that has disabled read concern "majority".

On replica set,

You can specify read concern "local" or "majority" or "snapshot" even in the replica set has disabled read concern “majority”.

However, if you are planning to transition to a sharded cluster with disabled read concern majority shards, you may wish to avoid using read concern "snapshot".

Tip

To check if read concern “majority” is disabled, You can run db.serverStatus() on the mongod instances and check the storageEngine.supportsCommittedReads field. If false, read concern “majority” is disabled.

For more information, see 3-Member Primary-Secondary-Arbiter Architecture and Three Member Primary-Secondary-Arbiter Shards.

Shard Configuration Restriction

You cannot run transactions on a sharded cluster that has a shard with writeConcernMajorityJournalDefault set to false (such as a shard with a voting member that uses the in-memory storage engine).

Diagnostics

MongoDB provides various transactions metrics:

For slow transactions (i.e. transactions that exceed the operationProfiling.slowOpThresholdMs threshhold), mongod and mongos log the transaction information under the TXN log component.

Feature Compatibility Version (FCV)

To use transactions, the featureCompatibilityVersion for all members of the deployment must be at least:

Deployment Minimum featureCompatibilityVersion
Replica Set 4.0
Sharded Cluster 4.2

To check the fCV for a member, connect to the member and run the following command:

db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

For more information, see the setFeatureCompatibilityVersion reference page.

Storage Engines

Starting in MongoDB 4.2, multi-document transactions are supported on replica sets and sharded clusters where:

  • the primary uses the WiredTiger storage engine, and
  • the secondary members use either the WiredTiger storage engine or the in-memory storage engines.

In MongoDB 4.0, only replica sets using the WiredTiger storage engine supported transactions.

Note

You cannot run transactions on a sharded cluster that has a shard with writeConcernMajorityJournalDefault set to false, such as a shard with a voting member that uses the in-memory storage engine.