Navigation
This is an upcoming (i.e. in progress) version of the manual.
  • Release Notes >
  • Release Notes for MongoDB 4.4 (Release Candidate)

Release Notes for MongoDB 4.4 (Release Candidate)

MongoDB 4.4 Release Candidates

While the 4.4 release candidates are available, these versions of MongoDB are for testing purposes only and not for production use.

Aggregation

Union All ($unionWith Stage)

MongoDB 4.4 adds the $unionWith aggregation stage, providing the ability to combines pipeline results from multiple collections into a single result set.

For details, see $unionWith.

Custom Aggregation Expressions

Starting in version 4.4, MongoDB provides the following new operators that allow users to define custom aggregation expressions:

With the addition of these new operators, you can use aggregation to write custom JavaScript expressions instead of relying on mapReduce and $where.

Note

Even before version 4.4, various map-reduce expressions could also be rewritten using other aggregation pipeline operators, such as $group, $merge, etc., without requiring custom functions.

For more information, see Map-Reduce to Aggregation Pipeline.

New Aggregation Operators

Operator Description
$accumulator Returns the result of a user-defined accumulator operator.
$binarySize Returns the size of a given string or binary data value’s content in bytes.
$bsonSize Returns the size in bytes of a given document (i.e. bsontype Object) when encoded as BSON.
$first Returns the first element in an array.
$function Defines a custom aggregation expression.
$last Returns the last element in an array.
$isNumber

Returns boolean true if the specified expression resolves to an integer, decimal, double, or long.

Returns boolean false if the expression resolves to any other BSON type, null, or a missing field

$replaceOne Replaces the first instance of a matched string in a given input.
$replaceAll Replaces all instances of a matched string in a given input.

General Aggregation Improvements

$out

Starting in MongoDB 4.4, $out can output to a collection in a different database. In earlier versions, $out can output to a collection to the same database where the aggregation is run.

$indexStats

Starting in MongoDB 4.4 (also available starting in 4.2.4), $indexStats includes the following fields in its output:

Field Description
shard Name of the shard, if applicable.
spec Index specification document
building A boolean flag that indicates if the index is currently being built.

$merge

Starting in MongoDB 4.4, $merge can output to the same collection that is being aggregated. You can also output to a collection which appears in other stages of the pipeline, such as $lookup.

Versions of MongoDB prior to 4.4 did not allow $merge to output to the same collection as the collection being aggregated.

Warning

When $merge outputs to the same collection that is being aggregated, documents may get updated multiple times or the operation may result in an infinite loop. This behavior occurs when the update performed by $merge changes the physical location of documents stored on disk. When the physical location of a document changes, $merge may view it as an entirely new document, resulting in additional updates. For more information on this behavior, see Halloween Problem.

$planCacheStats Changes

Starting in version 4.4,

Replica Sets

Resumable Initial Sync

Starting in MongoDB 4.4, a secondary performing initial sync can attempt to resume the sync process if interrupted by a transient (i.e. temporary) network error, collection drop, or collection rename. The sync source must also run MongoDB 4.4 to support resumable initial sync. If the sync source runs MongoDB 4.2 or earlier, the secondary must restart the initial sync process as if it encountered a non-transient network error.

By default, the secondary tries to resume initial sync for 24 hours. MongoDB 4.4 adds the initialSyncTransientErrorRetryPeriodSeconds server parameter for controlling the amount of time the secondary attempts to resume initial sync. If the secondary cannot successfully resume the initial sync process during the configured time period, it selects a new healthy source from the replica set and restarts the initial synchronization process from the beginning.

Prior to MongoDB 4.4, the secondary would restart the entire initial sync if it encountered an error during the process.

Rollback Directory

Starting in Mongo 4.4, the rollback directory for a collection is named after the collection’s UUID rather than the collection namespace; e.g.

<dbpath>/rollback/20f74796-d5ea-42f5-8c95-f79b39bad190/removed.2020-02-19T04-57-11.0.bson

For details, see Rollback Data.

Minimum Oplog Retention Period

Starting in MongoDB 4.4, you can specify the minimum number of hours to preserve an oplog entry. The mongod only removes an oplog entry if:

  • The oplog has reached the maximum configured size, and
  • The oplog entry is older than the configured number of hours based on the host system clock.

By default MongoDB does not set a minimum oplog retention period and automatically truncates the oplog starting with the oldest entries to maintain the configured maximum oplog size.

To configure the minimum oplog retention period when starting the mongod, either:

To configure the minimum oplog retention period on a running mongod, use replSetResizeOplog. Setting the minimum oplog retention period while the mongod is running overrides any values set on startup. You must update the value of the corresponding configuration file setting or command line option to persist those changes through a server restart.

Indexes Build Simultaneously on Data-Bearing Replica Set Members

Requires featureCompatibilityVersion 4.4+

Each mongod in the replica set or sharded cluster must have featureCompatibilityVersion set to at least 4.4 to start index builds simultaneously across replica set members.

MongoDB 4.4 running featureCompatibilityVersion: "4.2" builds indexes on the primary before replicating the index build to secondaries.

Starting with MongoDB 4.4, index builds on a replica set or sharded cluster build simultaneously across all data-bearing replica set members. For sharded clusters, the index build occurs only on shards containing data for the collection being indexed. The primary requires a minimum number of data-bearing voting members (i.e commit quorum), including itself, that must complete the build before marking the index as ready for use.

By default, index builds use a commit quorum of all data-bearing voting members. To start an index build with a non-default commit quorum, MongoDB 4.4 adds the commitQuorum parameter to createIndexes or its shell helpers db.collection.createIndex() and db.collection.createIndexes().

To modify the quorum required for an in-progress index build, MongoDB 4.4 introduces the new setIndexCommitQuorum command.

See Index Builds in Replicated Environments for more information.

Safe Replica Set Reconfiguration

Starting in MongoDB 4.4, replSetReconfig waits until a majority of members install the replica configuration before returning success. First, the operation waits until the current configuration is committed before installing the new configuration on the primary. The operation then waits until a majority of members install the new configuration before returning successfully. replSetReconfig waits indefinitely by default. MongoDB 4.4 adds the optional maxTimeMS parameter for specifying the maximum amount of time to wait for the operation to return successfully. See Safe Replica Set Reconfiguration for more information on the reconfiguration process.

Starting in MongoDB 4.4, the replSetGetConfig command can specify a new option commitmentStatus: true when run on the primary. When run with the option, the command includes in the output a commitmentStatus field. This output field indicates whether the replica set’s previous reconfig has been committed, so that the replica set is ready to be reconfigured again. For more information, see the replSetGetConfig command.

MongoDB 4.4 adds the term field to the replica set configuration document. Replica set members use term and version to achieve consensus on the “newest” replica configuration. Setting featureCompatibilityVersion (fCV) : “4.4” implicitly performs a replSetReconfig to add the term field to the configuration document and blocks until the new configuration propagates to a majority of replica set members. Similarly, downgrading to fCV : "4.2" implicitly performs a reconfiguration to remove the term field.

Preferred Initial Sync Source

Starting in MongoDB 4.4, you can specify the preferred initial sync source using the initialSyncSourceReadPreference parameter. You can only set this parameter on mongod startup, using either the setParameter configuration file setting or the --setParameter command line option.

initialSyncSourceReadPreference supports following read preference modes:

If the replica set has disabled chaining, the default initialSyncSourceReadPreference read preference mode is primary.

You cannot specify a tag set or maxStalenessSeconds to initialSyncSourceReadPreference.

Mirrored Reads

Starting in version 4.4, MongoDB provides mirrored reads to pre-warm electable secondary members’ cache with the most recently accessed data. With mirrored reads, the primary can mirror a subset of operations that it receives and send them to a subset of electable secondaries. Pre-warming the cache of a secondary can help restore performance more quickly after an election.

Note

The primary’s response to the client is not affected by the mirror reads. The mirrored reads are “fire-and-forget” operations by the primary; i.e., the primary does not await the response for the mirrored reads.

Mirrored Reads Parameter

MongoDB 4.4 adds the following mirrored reads parameter. You can set the parameter at startup using the setParameter configuration file setting or the --setParameter command line option or at runtime with setParameter command:

Parameter Description
mirrorReads

Specifies the samplingRate and maxTimeMS settings for mirrored reads.

{ samplingRate: <float>, maxTimeMS: <int> }

A samplingRate of 0 turns off mirrored reads.

Mirrored Reads Metrics

The command serverStatus and its corresponding mongo shell method db.serverStatus() return mirroredReads if you specify the field’s inclusion in the operation:

db.runCommand( { serverStatus: 1, mirroredReads: 1 } )

or

db.serverStatus( { mirroredReads: 1 } )

Sharded Clusters

Refinable Shard Keys

Starting in 4.4, MongoDB provides the refineCollectionShardKey command. With the new command, you can refine a collection’s shard key by adding a suffix field or fields to the existing key. Refining a collection’s shard key allows for a more fine-grained data distribution and can address situations where the existing key has led to jumbo (i.e. indivisible) chunks due to insufficient cardinality.

For example, you may have an existing orders collection with the shard key { customer_id: 1 }. You can change the shard key by adding a suffix order_id field to the shard key so that { customer_id: 1, order_id: 1 } becomes the new shard key, allowing data distribution by both customer_id and order_id fields.

To use the refineCollectionShardKey command, the sharded cluster must have feature compatibility version (fcv) of 4.4. For more information, see the refineCollectionShardKey command.

Note

After you refine the shard key, it may be that not all documents in the collection have the suffix field(s). To populate the missing shard key field(s), see Missing Shard Key.

Before refining the shard key, ensure that all or most documents in the collection have the suffix fields, if possible, to avoid having to populate the field afterwards.

In earlier versions, once you select a shard key, you cannot modify the shard key.

Missing Shard Keys

With the ability to refine a shard key with a suffix, it may be that not all documents in the collection have the suffix fields. Starting in version 4.4, documents in a sharded collection can be missing the shard key fields. In earlier versions, shard key fields must exist in every document for a sharded collection. For details, see Missing Shard Key.

Hedged Reads

To minimize latencies, mongos instances, by default, can use hedge reads. With hedged reads, the mongos instances can route read operations to multiple members per each queried shard and return results from the first respondent per shard. By default, mongos instances support using hedged reads. To turn off a mongos instance’s support for hedged reads, set the readHedgingMode parameter for the mongos.

Hedged reads are specified per operation as part of the read preference. Non-primary read preferences support hedged reads. Read preference nearest specifies hedged read by default.

For more information, see:

Hedged Read Parameters

Parameter Description
readHedgingMode Enables or disables mongos instance’s support for hedged reads.
maxTimeMSForHedgedReads Specifies the maximimum time limit (in milliseconds) for the additional read sent to hedge a read operation.

Hedged Read Option for Read Preference

To specify hedged read for a read preference, MongoDB 4.4 introduces the Hedged Read Option. To set using a MongoDB driver, refer to the driver read preference API documentation.

The following mongo shell methods can accept hedge options to enable hedged read for the specified read preference:

Hedged Read Metrics

The command serverStatus and its corresponding mongo shell method db.serverStatus() return hedgingMetrics.

balancerCollectionStatus Command

MongoDB 4.4 provides the command balancerCollectionStatus and the mongo shell helper method sh.balancerCollectionStatus() that return information about whether the chunks of a sharded collection are balanced (i.e. do not need to be moved) as of the time the command is run or need to be moved. With the command, users can verify that initial chunk creation and migration has finished.

Improved mongos Startup Procedure

Starting with MongoDB 4.4, mongos adds the following new default startup behavior:

  • mongos will now preload a sharded cluster’s routing table on startup, rather than doing so on-demand for the first incoming client connection.
  • mongos will now prewarm its connection pool to shard hosts on startup, rather than doing so on-demand for incoming client connections.

This behavior results in faster servicing of initial client connections after a mongos instance is started or restarted. In particular, this allows sites that employ multiple mongos instances to restart them as necessary, or add new ones, without initial client requests to those instances needing to wait on connection establishment.

Both routing table preloading and connection pool prewarming are enabled by default.

MongoDB 4.4 adds the following parameters for controlling this behavior:

Improved Routing Table Updates

Running flushRouterConfig is no longer required after executing the movePrimary or dropDatabase commands. These two commands now automatically refresh a sharded cluster’s routing table as needed when run. Manually issuing the flushRouterConfig command is still recommended in the cases described under flushRouterConfig Considerations.

Compound Hashed Shard Keys

Starting in MongoDB 4.4, you can shard a collection using a compound shard key with a single hashed field. Prior to 4.4, MongoDB did not support compound shard keys with a hashed field.

Compound hashed sharding supports features like zone sharding, where the prefix (i.e. first) non-hashed field or fields support zone ranges while the hashed field supports more even distribution of the sharded data. For example, the following operation shards a collection on a compound hashed shard key that supports zoned sharding:

sh.shardCollection(
  "examples.compoundHashedCollection",
  { "fieldA" : 1, "fieldB" : 1, "fieldC" : "hashed" }
)

Compound hashed sharding also supports shard keys with a hashed prefix for resolving data distribution issues related to monotonically increasing fields For example, the following operation shards a collection on a compound hashed shard key where the hashed field is the shard key prefix:

sh.shardCollection(
  "examples.compoundHashedCollection",
  { "_id" : "hashed", "fieldA" : 1}
)

General Sharded Clusters Improvements

Index Consistency Checks

Starting in MongoDB 4.4, the config server primary, by default, checks for index inconsistencies across the shards for sharded collections. The command serverStatus returns the field shardedIndexConsistency to report on index inconsistencies when run on the config server primary.

To configure the index consistency checks, MongoDB provides the following parameters:

Parameter Description
enableShardedIndexConsistencyCheck Enable or disable the index consistency checks.
shardedIndexConsistencyCheckIntervalMS The interval at which the config server’s primary checks the index consistency of sharded collections.

Concurrent removeShard Operations

Starting in MongoDB 4.4, you can have more than one removeShard operation in progress.

In earlier versions, removeShard returns an error if another removeShard operation is in progress.

Shard Key Limit

Starting in version 4.4, MongoDB removes the 512-byte limit on the shard key size. For MongoDB 4.2 and earlier, a shard key cannot exceed 512 bytes.

Partial Results

Starting in 4.4, if the find or subsequent getMore commands return partial results due to the unavailability of the queried shard(s), the output includes a boolean flag partialResultsReturned.

Jumbo Chunk Migration

For chunks that are too large to migrate, starting in MongoDB 4.4:

  • A new balancer setting attemptToBalanceJumboChunks allows the balancer to migrate chunks too large to move as long as the chunks are not labeled jumbo. See Balance Chunks that Exceed Size Limit for details.
  • The moveChunk command can specify a new option forceJumbo to allow for the migration of chunks that are too large to move. The chunks may or may not be labeled jumbo.

Automatically Split system.sessions Collection

Starting in version 4.4 (and 4.2.7), MongoDB automatically splits the system.sessions collection into at least 1024 chunks and distributes the chunks uniformly across shards in the cluster.

Transactions

Transactions Support Collection and Index Creation

Starting in MongoDB 4.4 with feature compatibility version (fcv) "4.4", you can:

  • Create collections inside of a multi-document transaction.
  • Create indexes inside of a multi-document transaction in the following scenarios:
    • The index is specified on a non-existing collection. In this case, the collection is created implicitly.
    • The index is specified on an empty collection which was created earlier in the same transaction.

For more details, see Create Collections and Indexes In a Transaction.

With fcv "4.2" or less, operations that affect the database catalog, such as creating or dropping a collection or an index, are not allowed in transactions. For details on transaction restrictions, see Restricted Operations.

Note

You can only run the createCollection() and createIndexes() methods when using read concern level of "local". If you specify a read concern level other than "local", the transaction fails.

Security Improvements

New Kerberos Validation Tool mongokerberos

MongoDB Enterprise 4.4 provides a new mongokerberos tool for validating your platform’s Kerberos configuration for use with MongoDB, and for testing end-to-end client authentication through Kerberos. When run, mongokerberos returns a report indicating any issues encountered, and provides potential advice for resolving them. mongokerberos is available in MongoDB Enterprise only.

OCSP

Starting in version 4.4, MongoDB enables, by default, the use of OCSP (Online Certificate Status Protocol) to check for certificate revocation. The use of OCSP eliminates the need to periodically download a Certificate Revocation List (CRL) and restart the mongod/mongos with the updated CRL.

In versions 4.0 and 4.2, the use of OCSP is available only through the use of system certificate store on Windows or macOS.

OCSP Stapling/Must Staple

As part of its OCSP support, MongoDB 4.4 supports the following on Linux:

  • OCSP stapling. With OCSP stapling, mongod and mongos instances attach or “staple” the OCSP status response to their certificates when providing these certificates to clients during the TLS/SSL handshake. By including the OCSP status response with the certificates, OCSP stapling obviates the need for clients to make a separate request to retrieve the OCSP status of the provided certificates.
  • OCSP must-staple extension. OCSP must-staple is an extension that can be added to the server certificate that tells the client to expect an OCSP staple when it receives a certificate during the TLS/SSL handshake.

OCSP Parameters

MongoDB 4.4 adds the following OCSP parameters. You can set these parameters at startup using the setParameter configuration file setting or the --setParameter command line option:

Parameter Description
ocspEnabled Enables or disables the OCSP support.
ocspValidationRefreshPeriodSecs Specifies the number of seconds to wait before refreshing the stapled OCSP status response.
tlsOCSPStaplingTimeoutSecs Specifies the maximum number of seconds the mongod/mongos instance should wait to receive the OCSP status response for its certificates.
tlsOCSPVerifyTimeoutSecs Specifies the maximum number of seconds that the mongod/mongos should wait for the OCSP response when verifying client certificates.

x.509 Certificates Nearing Expiry Trigger Warnings

Starting in MongoDB 4.4, mongod/mongos logs a warning on connection if the presented x.509 certificate expires within 30 days of the mongod/mongos system clock. Specifically, the following connections to a mongod or mongos can trigger x.509 certificate expiry warnings:

The warning log message resembles the following:

<Timestamp> W NETWORK [connection] Peer certificate <Certificate Subject Name> expires...

Consider proactively renewing client x.509 certificates nearing expiration to ensure continued connectivity to the cluster.

MongoDB 4.4 adds the tlsX509ExpirationWarningThresholdDays parameter for controlling certificate expiration warning threshold. Set the parameter to 0 to disable the warning. For complete documentation, see tlsX509ExpirationWarningThresholdDays.

TLS 1.3 Support

On CentOS 8 and RHEL 8, MongoDB 4.4 (as well as versions 4.2, 4.0, and 3.6) support TLS1.3.

Platform Support

  • MongoDB 4.4 removes support for:
    • Windows 7 / Server 2008 R2
    • Windows 8 / Server 2012
    • Windows 8.1 / Server 2012 R2
    • macOS 10.12

See Supported Platforms.

Tools

Migration to MongoDB Database Tools Project

Starting in MongoDB 4.4, the documentation for the following tools have been migrated to the MongoDB Database Tools project:

The MongoDB Database Tools use the Apache License, Version 2.0. See mongodb/mongo-tools for the source code.

Note

For documentation on previous versions of the listed tools, reference that version of the MongoDB server manual.

Quick links to older documentation:

New mongokerberos Kerberos Validation Tool

MongoDB Enterprise 4.4 provides a new mongokerberos tool for validating your platform’s Kerberos configuration for use with MongoDB, and for testing end-to-end client authentication through Kerberos. When run, mongokerberos returns a report indicating any issues encountered, and provides potential advice for resolving them. mongokerberos is available in MongoDB Enterprise only.

See the mongokerberos reference page for more information.

mongoreplay Removed from MongoDB Packaging

Starting in MongoDB 4.4, mongoreplay is removed from MongoDB packaging. mongoreplay and its related documentation are migrated to the mongodb-labs github project. Projects in mongodb-labs are experimental and not officially supported by MongoDB.

Quick links to older documentation

Drivers

New Drivers

Indexes

Compound Hashed Indexes

MongoDB 4.4 adds support for creating compound indexes with a single hashed field. MongoDB 4.2 and earlier only supported single field hashed indexes.

The following operation creates a compound hashed index on country and _id:

db.examples.createIndex( { "country" : 1, "_id" : "hashed" } )

Compound hashed indexes require featureCompatibilityVersion set to 4.4.

Hidden Indexes

Starting in version 4.4, MongoDB adds the ability to hide or unhide indexes from the query planner. An index hidden from the query planner is not evaluated as part of query plan selection.

By hiding an index from the planner, users can evaluate the potential impact of dropping an index without having to drop the index. If the impact is negative, the user can unhide the index instead of having to recreate a dropped index. And because indexes are fully maintained while hidden, hidden indexes are immediately available for use once unhidden.

For details, see Hidden Indexes.

To support hidden indexes, MongoDB introduces:

dropIndexes Can Abort In-Progress Index Builds

If an index specified to dropIndexes is still building, dropIndexes attempts to abort the in-progress build. Aborting an index build has the same effect as dropping the built index. Prior to MongoDB 4.4, dropIndexes would return an error if the collection had any in-progress index builds. This behavior also applies to the shell helpers db.collection.dropIndex() and db.collection.dropIndexes().

To drop a specific index out of a set of related in-progress builds, wait until the index builds complete and specify that index to dropIndexes or its shell helpers.

For more complete documentation, see:

drop() Can Abort In-Progress Index Builds

Starting in MongoDB 4.4, the db.collection.drop() method and drop command abort any in-progress index builds on the target collection before dropping the collection. Prior to MongoDB 4.4, attempting to drop a collection with in-progress index builds results in an error, and the collection is not dropped.

For replica sets or shard replica sets, aborting an index on the primary does not simultaneously abort secondary index builds. MongoDB attempts to abort the in-progress builds for the specified indexes on the primary and if successful creates an associated abort oplog entry. Secondary members with replicated in-progress builds wait for a commit or abort oplog entry from the primary before either committing or aborting the index build.

For replica sets or shard replica sets, aborting an index on the primary does not simultaneously abort secondary index builds. MongoDB attempts to abort the in-progress builds for the specified indexes on the primary and if successful creates an associated abort oplog entry. Secondary members with replicated in-progress builds wait for a commit or abort oplog entry from the primary before either committing or aborting the index build.

dropDatabase Can Abort In-Progress Index Builds

Starting in MongoDB 4.4, the db.dropDatabase() method and dropDatabase command abort any in-progress index builds on collections in the target database before dropping the database. Aborting an index build has the same effect as dropping the built index. Prior to MongoDB 4.4, attempting to drop a database that contains a collection with an in-progress index build results in an error, and the database is not dropped.

Deprecation of geoHaystack Index and geoSearch Command

MongoDB 4.4 deprecates the geoHaystack index and the geoSearch command. Use a 2d index with $geoNear or $geoWithin instead.

Removed Commands

MongoDB removes the following command(s) and mongo shell helper(s):

Removed Command Removed Helper Alternatives
cloneCollection db.cloneCollection()
planCacheListPlans PlanCache.getPlansByQuery()
planCacheListQueryShapes PlanCache.listQueryShapes()

Networking

Support for TCP Fast Open

Starting with MongoDB 4.4, mongod and mongos support TCP Fast Open (TFO) connections by default. TFO requires both the client and the mongod/mongos host machines support and enable TFO:

Windows

The following Windows operating systems support TFO:

  • Microsoft Windows Server 2016 or later.
  • Microsoft Windows 10 Update 1607 or later.
macOS
macOS 10.11 (El Capitan) and later support TFO
Linux

Linux operating systems running Linux Kernel 3.7 or later can support inbound TFO connections.

Linux operating systems running Linux Kernel 4.11 or later can support both inbound and outbound TFO connections.

Set the value of /proc/sys/net/ipv4/tcp_fastopen to enable support for inbound and/or outbound TFO connections:

  • Set to 1 to enable only outbound TFO connections
  • Set to 2 to enable only inbound TFO connections
  • Set to 3 to enable inbound and outbound TFO connections.

MongoDB 4.4 adds the following parameters for controlling TFO:

Parameter Description
tcpFastOpenServer

Default: true (Enabled)

Enables or disables support for inbound TFO connections to the mongod/mongos

tcpFastOpenClient

Default: true (Enabled)

Linux Operating System Only

Enables or disables support for outbound TFO connections from the mongod/mongos.

tcpFastOpenQueueSize

Default: 1024

Control the size of the queue of pending TFO connections.

MongoDB 4.4 adds the following counters to the output of serverStatus and db.serverStatus():

Counter Description
network.tcpFastOpen.kernelSetting

Linux only

Indicates kernel support for TFO.

network.tcpFastOpen.serverSupported Indicates operating system support for incoming TFO connections.
network.tcpFastOpen.clientSupported Indicates operating system support for outgoing TFO connections.
network.tcpFastOpen.accepted Indicates the total number of accepted incoming TFO connections to the mongod/mongos since the mongod/mongos last started.

A complete discussion of TFO is outside the scope of this documentation. For more information on TFO, start with the following external resources:

Validation

Support for Background Validation

Starting in MongoDB 4.4, for the WiredTiger storage engine, you can run the validate command in the background or in the foreground. In earlier versions, you can only run validate in the foreground:

  • When run in background, the operation takes an intent shared lock on the collection and does not block reads and writes or DDL operations on the collection; however, the operation does not update collection count and size metrics.
  • When run in foreground, the operation takes an exclusive lock on the collection and blocks all reads and writes and DDL operations on the collection until the operation finishes. When run on a secondary, the operation can block all other operations on that secondary until it finishes.

For details, see the validate command and the db.collection.validate() helper.

maxValidateMBperSec Parameter

MongoDB 4.4 adds the maxValidateMBperSec parameter to limit the I/O and CPU usage when running background validation. See maxValidateMBperSec for details.

Data Throughput Information

Starting in version MongoDB 4.4,

General Improvements

Blocking Sort Limit Increased

If MongoDB cannot use an index or indexes to obtain the sort order for a given cursor.sort() operation, MongoDB must perform a blocking sort on the data. A blocking sort indicates that MongoDB must consume and process all input documents to the sort before returning results. Blocking sorts do not block concurrent operations on the collection or database.

Prior to MongoDB 4.4, MongoDB returned an error if a blocking sort operations required more than 32 megabytes of system memory. Starting in MongoDB 4.4, blocking sort operations increase the limit on system memory to use for the sort operation to 100 megabytes. For blocking sort operations which require more than 100 megabytes of system memory, MongoDB returns an error unless the query specifies cursor.allowDiskUse() (New in MongoDB 4.4).

For more information on sorting and index use, see Sort and Index Use.

find Can Use Temporary Files To Support Large Non-Indexed Sorts

MongoDB 4.4 adds a new option allowDiskUse to the find command. With allowDiskUse: true, the operation can use temporary files on disk when processing a non-indexed (“blocking”) sort operation that exceeds the 100 megabyte memory limit. Prior to MongoDB 4.4, a find operation with a blocking sort failed if it exceeded the memory limit while processing the sort.

For the db.collection.find() shell method with cursor.sort(), MongoDB 4.4 adds the cursor.allowDiskUse() cursor modifier.

allowDiskUse and cursor.allowDiskUse() have no effect if MongoDB can satisfy the sort using an index, or if the blocking sort requires less than 100 megabytes of memory.

For instructions on enabling allowDiskUse for queries issued through a MongoDB driver, defer to the documentation for your preferred MongoDB 4.4-compatible driver.

Collection Namespace Limit

Starting in MongoDB 4.4,

  • For featureCompatibilityVersion set to "4.4" or greater, MongoDB raises the limit on collection/view namespace to 255 bytes. For a collection or a view, the namespace includes the database name, the dot (.) separator, and the collection/view name (e.g. <database>.<collection>),
  • For featureCompatibilityVersion set to "4.2" or earlier, the maximum length of the collection/view namespace remains 120 bytes.

serverStatus Output Change

Field Name Change

serverStatus returns flowControl.locksPerKiloOp instead of flowControl.locksPerOp.

db.auth() Can Prompt for Password

Starting in MongoDB 4.4, the mongo shell method db.auth(<username>, <password>) prompts for the password if you do not pass in the password or the passwordPrompt() method for the <password>.

Support for $natural Sort on Views

Starting in MongoDB 4.4, you can specify a $natural sort when running a find operation against a view.

$meta Operator

$meta Keyword Support
Starting in MongoDB 4.4, the $meta operator adds support for retrieving the indexKey metadata. The indexKey metadata is for debugging purposes only and not for application logic. See $meta for more information.
{ $meta: "textScore" } Usage with find()

Starting in version 4.4, MongoDB makes the following { $meta: "textScore" } changes when used with db.collection.find():

  • You must specify the $text operator in the query predicate to use { $meta: "textScore" }.

  • You can sort the resulting documents by their search relevance, i.e. { $meta: "textScore" }, without also having to project the textScore.

    In earlier versions, to include { $meta: "textScore" } expression in the sort(), you must also include the same expression in the projection.

  • If you include the { $meta: "textScore" } expression in both the projection and sort, i.e. db.collection.find(<$text search>, <projection>).sort(<sort>), the projection and sort documents can have different field names for the expression.

    In previous versions of MongoDB, if you include the { $meta: "textScore" } in both the projection and sort, you must specify the same field name in both places.

For more information, see Text Score Metadata $meta: "textScore". For examples of "textScore" projections and sorts, see Text Search Score Examples.

Support for Diagnostic Backtrace Generation

Starting in MongoDB 4.4, mongod and mongos processes running on Linux will now log a backtrace for each of their running threads upon receipt of a SIGUSR2 signal. This backtrace can be analyzed for diagnostic information or provided to MongoDB support as needed. This functionality is currently available only on the x86_64 architecture. For more information on using this feature, see Generate a Backtrace.

Container-aware FTDC Reporting

Starting in MongoDB 4.4, FTDC now reports utilization data for a mongod running in a container from the perspective of the container, as opposed to the host operating system. See Full Time Diagnostic Data Capture for more information.

Updated ulimit Startup Warning

Starting in MongoDB 4.4, mongod will log a startup warning if a platform’s configured ulimit value for number of open files is under 64000. Previously, a warning would only be logged if this value was under 1000. See Recommended ulimit Settings for more information.

New replanReason Database Profiler Output Field

MongoDB 4.4 adds the replanReason field to database profiler output and diagnostic log messages. The replanReason field contains the reason the query system evicted a cached plan.

dbStats and collStats Output

The dbStats command and its mongo shell helper db.stats() return:

The collStats command, its mongo shell helper db.collection.stats(), and the $collStats aggregation stage return:

Delete and Hint

Starting in MongoDB 4.4, the delete command and the associated mongo shell methods db.collection.deleteOne() and db.collection.deleteMany() can accept a hint argument to specify the index to use.

Additionally, starting in MongoDB 4.4, when a findAndModify command has remove set to true, the command can accept a hint argument to specify the index to use.

See:

JavaScript Execution on mongos

Starting in MongoDB 4.4, MongoDB allows JavaScript execution on mongos instances. To disable JavaScript execution on a mongos instance:

Earlier versions of MongoDB do not allow JavaScript execution on mongos instances.

Global Default Read and Write Concern

Requires featureCompatibilityVersion 4.4+

Each mongod in the replica set or sharded cluster must have featureCompatibilityVersion set to at least 4.4 to configure global default read and write concern.

Starting in MongoDB 4.4, replica sets and sharded clusters support configuring global default read and write concern settings. Clients which do not explicitly specify a given read or write concern setting inherit the corresponding global default setting.

To configure default global default read or write concern, MongoDB adds the setDefaultRWConcern administrative command. For replica sets, issue the command against the primary member. For sharded clusters, issue the command from a mongos.

To retrieve the global default read or write concern settings, MongoDB adds the getDefaultRWConcern administrative command.

Read Concern Provenance

Starting in MongoDB 4.4, read concern objects may include a provenance field, indicating where the read concern originated.

The following table shows the possible read concern provenance values and their significance:

Provenance Description
clientSupplied The read concern was specified in the application.
customDefault The read concern originated from a custom defined default value. See setDefaultRWConcern.
implicitDefault The read concern originated from the server in absence of all other read concern specifications.

If a read operation is logged or profiled, the operation entry contains the read concern object, including the provenance field.

MongoDB does not recommended specifying the provenance field in requests to the server. This field should only be used for diagnostic purposes.

Write Concern Provenance

Starting in MongoDB 4.4, write concern objects may include a provenance field, indicating where the write concern originated.

The following table shows the possible write concern provenance values and their significance:

Provenance Description
clientSupplied The write concern was specified in the application.
customDefault The write concern originated from a custom defined default value. See setDefaultRWConcern.
getLastErrorDefaults The write concern originated from the replica set’s settings.getLastErrorDefaults field.
implicitDefault The write concern originated from the server in absence of all other write concern specifications.

If a write operation is logged or profiled, the operation entry contains the write concern object, including the provenance field.

MongoDB does not recommended specifying the provenance field in requests to the server. This field should only be used for diagnostic purposes.

currentOp Output

New KMIP Connection Parameters for mongod

MongoDB 4.4 Enterprise introduces two new configuration settings to enhance the initial connection to a KMIP server, as part of Kerberos authentication:

Connection Retries

To control the number of times the mongod retries a failed initial connection to the KMIP server:

Connection Timeout

To control the timeout, in milliseconds, to wait for the initial response from the KMIP server before giving up, or retrying:

These settings are available in MongoDB Enterprise only.

Changes Affecting Compatibility

Some changes can affect compatibility and may require user actions. For a detailed list of compatibility changes, see Compatibility Changes in MongoDB 4.4 (Release Candidate).

Upgrade Procedures

Feature Compatibility Version

To upgrade from 4.2 deployment, the 4.2 deployment must have featureCompatibilityVersion set to 4.2. To check the version:

db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

For specific details on verifying and setting the featureCompatibilityVersion as well as information on other prerequisites/considerations for upgrades, refer to the individual upgrade instructions:

If you need guidance on upgrading to 4.4, MongoDB offers major version upgrade services to help ensure a smooth transition without interruption to your MongoDB application.

Report an Issue

To report an issue, see https://github.com/mongodb/mongo/wiki/Submit-Bug-Reports for instructions on how to file a JIRA ticket for the MongoDB server or one of the related projects.