Navigation

4.0 Changelog

4.0.4 Changelog

Security

SERVER-37135 TLSVersionCounts needs to track and report TLS 1.3

Sharding

  • SERVER-29160 Sharding commonly uses write concern timeouts of 15 seconds and these are timing out in migration related operations and causing BFs
  • SERVER-31563 Reevaluate not_allowed_on_sharded_collection_cmd.js testing
  • SERVER-31892 moveChunk with waitForDelete doesn’t wait for majority write concern
  • SERVER-35323 sessionId matching ignores userId part of the lsid
  • SERVER-35763 lastWriteDate field can get out of sync between repl set nodes during migration
  • SERVER-36831 LogicalSessionCache on mongos does not correctly report active operations
  • SERVER-36850 Add a replication passthrough suite to detect errors in LogicalSessionsCache
  • SERVER-36959 _shardServerShardCollection should use afterOpTime when counting chunks on the config server
  • SERVER-37142 invariant failure during movePrimary during cleanup
  • SERVER-37330 Add sharded passthrough suites to detect errors in LogicalSessionCache
  • SERVER-37430 Destroy sharding task executors and AsyncRequestSenders after the PeriodicRunner is destroyed in mongod shutdown
  • SERVER-37496 The balancer shouldn’t register a shutdown task after the process is multithreaded
  • SERVER-37578 Assert that a zone is associated with a shard before sharding the collection
  • SERVER-37657 Report the offending oplog entries if a batch contains non-increasing transaction numbers
  • SERVER-37918 Fast initial split algorithm produces corrupted routing info if passed unsorted zones list

Replication

  • SERVER-20845 re-add replSetReconfig to auditing suite
  • SERVER-36978 TaskRunner must ensure Client is initialized for thread before running tasks
  • SERVER-36979 Aborting a transaction must abort WUOW before releasing locks.
  • SERVER-37118 Coverity analysis defect 105000: Arguments in wrong order
  • SERVER-37227 Reintroduce enableMajorityReadConcern:false server parameter
  • SERVER-37514 Snapshot readConcern without atClusterTime should always be speculative

Query

  • SERVER-37058 Update with numeric field names inside an array can cause validation to fail
  • SERVER-37132 Negation of $in with regex can incorrectly plan from the cache, leading to missing query results

JavaScript

  • SERVER-30773 bsonWoCompare and bsonBinaryEqual should work with bsonelement types
  • SERVER-37126 Invoke runSafely for all external implscope methods

Storage

  • SERVER-26854 LockStats for sub-operations should not include time for previous sub ops
  • SERVER-36534 Don’t acquire locks on oplog when writing oplog entries
  • SERVER-36883 support non-doc-locking storage engines in SERVER-36534
  • SERVER-37055 IndexBuildBlock::fail() should grab a lock
  • SERVER-37394 Invariant failure deadline != Date_t::max() or Invariant failure date.isFormattable()
  • SERVER-37618 Capture all the logs in lock_stats_suboperation_logs.js
  • SERVER-37662 Include backupCursor state in output of serverStatus
  • SERVER-37749 replSetResizeOplog command does not validate argument

Build and Packaging

  • SERVER-37067 Upgrade static OpenSSL to 1.1.0i
  • SERVER-37158 Set compatibility_version field for darwin SDK shared libraries
  • SERVER-37402 Improve mobile licensing text
  • SERVER-37407 Harmonize embedded tarball name and unpacked directory names
  • SERVER-37488 Install BCSymbolsMap into darwin embedded frameworks
  • SERVER-37584 Build failure on case-sensitive macOS file system: CommonHmac.h
  • SERVER-37596 Add debug info packages to mobile SDK builds
  • SERVER-37640 no member named ‘SSLCopyRequestedPeerNameLength’ in the global namespace
  • SERVER-37651 Update license files for MongoDB Community Edition
  • SERVER-37691 Compile WatchOS builds with -fapplication-extension flag
  • SERVER-37754 Duplicate license headers in IDL files
  • SERVER-37785 Rename bundle identifier in frameworks to adhere to bundle naming rules
  • SERVER-37852 Publish mobile artifacts on release builds

Tools

  • TOOLS-2069 mongoreplay does not support SCRAM-SHA-256
  • TOOLS-2102 Mongorestore does not check for errors decoding the oplog.bson file
  • TOOLS-2131 mongorestore hang in replaying oplog with –archive and –oplogReplay option

Internals

  • SERVER-18985 setParameter should log at level 0
  • SERVER-31570 Adjust mongobridge port allocations for easier debugging
  • SERVER-33470 Log archival message, even if successful, in hook_test_archival.py
  • SERVER-34986 CIDR Block That is Exempt from maxConns
  • SERVER-35570 Improve robustness of backup_restore.js around dropping test database
  • SERVER-35818 provide a stdx::variant
  • SERVER-35861 Remove call to dashboard_gen.py in perf.yml
  • SERVER-36250 Add support for optionally logging specific negotiated TLS versions
  • SERVER-36420 SecTrustCopyAnchorCertificates is not safe to use after a fork
  • SERVER-36451 ContinuousStepdown with killing nodes can hang due to not being able to start the primary
  • SERVER-36705 Add CocoaPod for embedded darwin frameworks
  • SERVER-36721 list_local_sessions.js cannot run concurrently with refreshLogicalSessionCacheNow
  • SERVER-36756 Log the githash of the 10gen/jstestfuzz repository when the fuzzer’s self-tests fail
  • SERVER-36885 Make ASIO remember IOCP state when transiently out of resources
  • SERVER-36964 Prevent secondaries in SessionsCollectionRS from attempting to set up the sessions collection.
  • SERVER-36986 list_local_sessions.js expects non-existent session to exist when running concurrently with LogicalSessionsCache refresh
  • SERVER-36988 awaitdata_getmore_cmd.js times out when run concurrently with the LogicalSessionCache refresh suite
  • SERVER-37064 Wrap “mongod_flags” onto multiple lines for readability
  • SERVER-37081 Catch asio::system_errors when resolving GenericSocket
  • SERVER-37083 Improve Window secure allocator
  • SERVER-37228 Escape double quotes in hang analyzer’s waitsfor graph
  • SERVER-37334 Strip .jar and .aar of unnecessary shared objects
  • SERVER-37353 Handle $slice value of LLONG_MIN gracefully
  • SERVER-37391 plan_cache_index_create.js should wait for index build start, rather than just createIndexes command start
  • SERVER-37393 Fix destructor race in ReplicaSetMonitorManager
  • SERVER-37410 Add standalone passthrough suites to detect errors in LogicalSessionCache
  • SERVER-37411 Fix bundle identifiers in embedded Info.plist
  • SERVER-37424 Sys-perf: change v4.0 branch batch time to once a week
  • SERVER-37425 Longevity tests – increase batch time to once a year
  • SERVER-37464 Reduce threadCount and iterations in secondary_reads.js workload
  • SERVER-37467 Have collect_resource_info.py recover from transient errors.
  • SERVER-37477 Disable TIG daily cron for update_test_lifecycle
  • SERVER-37495 Change Android minimum API level to 21
  • SERVER-37513 Pass along JNA exceptions from MongoEmbeddedCAPI.create
  • SERVER-37561 Server startupWarnings show spurious empty lines
  • SERVER-37563 Get rid of getGlobalAuthorizationManager and unnecessary references to the authorization_manager_global library
  • SERVER-37583 Changes to etc/cloud_nightly.yml
  • SERVER-37599 Log exit code of shell-spawned processes
  • SERVER-37683 Change embedded iOS min version to 11.0 in CocoaPod podspec
  • SERVER-37684 Change embedded iOS min version to 11.0 in evergreen builders
  • SERVER-37685 ensure free monitoring queue preserves FIFO on messages with same deadline
  • SERVER-37701 Make SessionUpdateTracker include the uid portion of LogicalSessionId when tracking
  • SERVER-37744 Fix license path in mongoc_embedded podspec
  • SERVER-37755 Invalid .plist for mongoc and bson .frameworks
  • SERVER-37813 Switch Android publishing from Bintray to Artifactory
  • SERVER-37832 Fix Windows/OSX ssl_options_test unittests on v4.0
  • WT-3898 Keep prepared updates in lookaside until they are read
  • WT-3995 Enhance timestamp abort to accept more number of threads
  • WT-4149 Log recovery and salvage should handle removal or truncation of log files
  • WT-4164 Ensure test/format configures a reasonably sized cache
  • WT-4214 Simplify timestamp handling for timestamp abort test
  • WT-4217 Enhance commit and rollback to re-read prepared updates
  • WT-4220 Enable long running prepared support
  • WT-4224 Add statistics for prepared transactions
  • WT-4293 WT_CURSOR.remove can lose a cursor position
  • WT-4297 Enhance steady throughput workload
  • WT-4314 Don’t access unmapped pages when evicting a tree
  • WT-4315 In rollback_to_stable, only check timestamp order if enforced
  • WT-4322 Enable direct I/O based crash test in make check
  • WT-4323 Fix race between setting transaction read_timestamp and updating global pinned timestamp
  • WT-4328 Use an internal session handle for schema operations in a txn
  • WT-4330 Change the wt utility to not use stdout if the -f option to the dump or printlog commands is specified
  • WT-4333 WiredTiger cursor cache doesn’t handle all possible locked handle states
  • WT-4335 Don’t fail rollback_to_stable due to sweep activity
  • WT-4337 Coverity #1395811 Time of check time of use
  • WT-4338 New WT_TXN_TS_XXX flags break #undef HAVE_TIMESTAMPS build
  • WT-4339 Revert part of a previous commit that simplified handle locking
  • WT-4340 The cursor caching layer can incorrectly release too many handle locks
  • WT-4341 Support million collection testing in Evergeen
  • WT-4342 Set session max on all configuration strings in timestamp_abort test
  • WT-4346 Remove prepared updates from lookaside on reading the page.
  • WT-4347 Limit the threads spawned by timestamp_abort with default config
  • WT-4348 Create all tables before spawning threads in random_directio test
  • WT-4351 Ensure resolving prepared transactions use updates from itself
  • WT-4355 Fail to find prepared updates during transaction rollback
  • WT-4358 Enhance the handle-lock stress test program
  • WT-4374 Fix a bug where a page could transition from WT_REF_LIMBO incorrectly
  • WT-4387 Fix ordering of referenced shared libraries in workgen
  • WT-4389 Update wtperf runner script to accept multiple arguments

4.0.3 Changelog

Security

  • SERVER-35418 Allow specifying CAs for incoming and outgoing connections separately
  • SERVER-36456 MongoD does not support kerberos on Windows
  • SERVER-36827 Warning about “No SSL certificate validation can be performed” is misleading

Sharding

  • SERVER-30841 Lower the amount of metadata refresh logging
  • SERVER-34500 Use causal consistency in database_versioning_upgrade_downgrade.js test when reading from secondary after setFCV
  • SERVER-35222 Crash on the config server at expired session cleanup
  • SERVER-35755 CollectionLock acquisition in shard_filtering_metadata_refresh.cpp can cause server to terminate on stepdown
  • SERVER-35773 MetadataManager directly calls into the CatalogCache
  • SERVER-36054 Get rid of ScopedCollectionMetadata’s operator bool
  • SERVER-36116 Get rid of CollectionShardingState::resetAll
  • SERVER-36130 Migration status reports verify connection strings – but those can change
  • SERVER-36164 Decouple ScopedCollectionMetadata from MetadataManager
  • SERVER-36332 CursorNotFound error in GetMore on a secondary with sessions
  • SERVER-36433 NamespaceSerializer lock should be used during drop database
  • SERVER-36634 Change FCV check to check for CommandNotFound in _shardsvrShardCollection and write to config after creating collections on non-primary shards
  • SERVER-37050 wait for replication in session_collection_auto_healing

Replication

  • SERVER-35616 Oplog query on initial syncing node can cause segmentation fault
  • SERVER-35793 Write concurrency test to verify server-wide transactions metrics tracking
  • SERVER-35821 readConcern:snapshot transactions need a read timestamp <= WT’s all_committed point
  • SERVER-36127 use w:majority instead of w:2 in apply_batch_only_goes_forward.js
  • SERVER-36470 Prevent change_stream_failover.js to elect a new primary which is the same as old primary after old primary steps down.
  • SERVER-36503 Skip dry-run election during election handoff
  • SERVER-36539 Test that DBClientCursor sends getMore for exhaust cursor if moreToCome is not set
  • SERVER-36565 Killing a session with a transaction can cause the thread to throw WriteConflictError
  • SERVER-36746 A failed step down attempt shouldn’t unconditionally reset LeaderMode to kMaster
  • SERVER-36846 TransactionsMetrics unit test should sleep a bit when trying to assert elapsed time > 0
  • SERVER-36975 Fix race condition in read_concern_snapshot_catalog_invalidation.js
  • SERVER-36982 Reintroduce enableMajorityReadConcern:false server parameter
  • SERVER-36985 Test single replica set transactions immediately after a rollbackViaRefetch
  • SERVER-37010 Prevent unexpected elections in initial_sync4.js and initial_sync_rename_collection*.js
  • SERVER-37048 Hold global intent lock whenever accessing the oplog collection pointer
  • SERVER-37105 Make it clear from the stack trace whether a command is running in a transaction
  • SERVER-37147 sessions_collection_auto_healing.js should use 2 node replica set
  • SERVER-37152 Increase write concern timeouts for writes expected to succeed in tags.js

Query

  • SERVER-13946 Consider putting skip stages below fetch stages
  • SERVER-36212 getMore should not enforce that session id matches until FCV is 4.0
  • SERVER-36299 Implement support for exhaust cursors with OP_MSG in DBClientCursor
  • SERVER-36435 Increase maxTimeMS timeout in awaitdata_getmore_cmd.js
  • SERVER-36453 Fix race condition in aggregation_cursor_invalidations.js
  • SERVER-36944 applyOps does not permit unknown field names when creating a v:1 index
  • SERVER-36951 applyOps should work with a createIndexes command without a UUID

Aggregation

SERVER-36993 mongod crash: Invariant failure indexedOr src/mongo/db/query/index_tag.cpp 237

Storage

  • SERVER-34577 read_after_optime.js fails on mongoe
  • SERVER-34606 Test (and possibly fix) behavior around majority commit point and oplog truncation
  • SERVER-35657 Do not delay journal flushes when operations are waiting for oplog visibility
  • SERVER-35780 renameCollection across databases incorrectly timestamps metadata for secondary index builds
  • SERVER-36400 Explicitly destroy the client on exiting the run body of each BackgroundJob
  • SERVER-36531 Lock acquisition may throw despite presence of UninterruptibleLockGuard when WT tickets are exhausted
  • SERVER-36879 write regression test for stuck cache issue during rollback
  • SERVER-36961 createIndexes command should check if index already exists with weak lock
  • SERVER-36969 initial_sync_wt_cache_full.js takes too long to complete on slow hosts
  • SERVER-37002 dropping a collection with long index names via rename fails under MMAPv1
  • SERVER-37121 Retry timestamping secondary background index builds

WiredTiger

  • SERVER-19815 Make repair more robust with the WiredTiger storage engine

Operations

  • SERVER-27588 Warning to disable defrag of transparent_hugepages when thp is disabled
  • SERVER-33606 mongo shell startSession() should fail if the server doesn’t support logical sessions
  • SERVER-34864 String-valued fields should not participate in schema change detection in ftdc
  • SERVER-35989 Mongo shell needs a countDocuments function

Build and Packaging

Tools

  • SERVER-30997 mongo cli –password is masked, but not when using mongodb:// connection string
  • TOOLS-2035 mongofiles_write_concern_mongos.js fails on server unstable

Internals

  • SERVER-28990 when started with –repair mongod should not try to bind to a port
  • SERVER-33908 Add pre and post invocation background activity hooks to CAPI implementation
  • SERVER-33978 References to sudo in evergreen.yml should use ${set_sudo}
  • SERVER-34120 scoped connection not being returned to the pool
  • SERVER-34711 Enable burn_in_tests to understand Evergreen task selectors
  • SERVER-34798 Replace subclasses of ServiceContext with decorations and flexible initialization code
  • SERVER-35216 Expose ReplicaSetManager refresh period to the mongo shell
  • SERVER-35233 Powercycle remote collection validation does not skip views
  • SERVER-35284 C++ “death test” unit tests should fork before constructing the test fixture, not after.
  • SERVER-35517 Add failpoint mechanism to the mongo shell
  • SERVER-35585 Make PeriodicRunner jobs be pausable/resumable
  • SERVER-35629 Use WiredTiger salvage API for repairing metadata files
  • SERVER-35630 Missing or corrupt data files should cause MongoDB to exit with an error message to run repair
  • SERVER-35696 Provide a way to restore idents known by WiredTiger, but unknown to the _mdb_catalog
  • SERVER-35731 Prevent a repaired node from re-joining a replica set
  • SERVER-35782 Repair should move aside unsalvageable data files and create empty ones in their place
  • SERVER-35784 Fix ambiguously named test fixture in commands_test.cpp
  • SERVER-35800 resmoke.py should retry getting a build_id and test_id from logkeeper
  • SERVER-35985 sessions_test and sharding_catalog_manager_test don’t destroy all Clients before destroying the ServiceContext
  • SERVER-36019 Create script to collect resource utilization of Android application
  • SERVER-36069 Vendor mongoebench-compatible JSON config files from mongodb/mongo-perf into src/third_party
  • SERVER-36076 Create new resmoke.py test suite for running mongoebench on a desktop
  • SERVER-36077 Create new resmoke.py test suite for running mongoebench on an Android device
  • SERVER-36078 Integrate adb resource monitor into mongoebench test suite for Android
  • SERVER-36084 Remove sharding runtime from embedded
  • SERVER-36162 Powercycle - ensure internal crash command has been executed on the remote host
  • SERVER-36169 Resmoke: bare raise outside except in the stepdown hook
  • SERVER-36258 Perform construction of ServiceContext after execution of mongo initializers, not during
  • SERVER-36347 Make parse_zone_info.js handle new error message from ServiceContext refactor
  • SERVER-36351 ServiceContextMongoDTest doesn’t persist TempDir correctly
  • SERVER-36474 Cannot initiate a replica set if free monitoring is disabled at command-line
  • SERVER-36621 A call to log() in capi_test.cpp does not print to stdout
  • SERVER-36691 Only recover orphaned collection idents when starting with repair, not after an unclean shutdown
  • SERVER-36702 SCons to be able to install binaries as darwin framework
  • SERVER-36703 SCons to be able to link against darwin frameworks
  • SERVER-36704 Embedded to support logical sessions the same way as standalone mongod
  • SERVER-36722 Add remote maven repository for embedded AAR/JAR
  • SERVER-36725 periodic_runner_impl_test should manually call tearDown
  • SERVER-36732 Wait for all secondaries to be up and electable in election handoff jstests
  • SERVER-36747 Add a small time delay to “jstests/ssl/ssl_client_certificate_warning_suppression.js”
  • SERVER-36749 Remove race in Service Executor Adaptive shutdown
  • SERVER-36757 Generate and extract mongoebench-compatible JSON config files to consistent locations
  • SERVER-36761 Add requires_replication tag to disk/repair_invalidates_replica_set_config.js
  • SERVER-36768 Orphaned collection recovery should allow recovered collections to be renamed
  • SERVER-36783 Run the secondary_reads_passthrough task of enterprise-rhel-62-64-bit-inmem on rhel62-large
  • SERVER-36836 v4.0 fsm workload “yield_group.js” should accept “InternalError” as a possible outcome of the group command
  • SERVER-36842 Core dump not generated after invariant failure in powercycle hosts
  • SERVER-36869 fsm capped collection checks should only make 1 query for assertions
  • SERVER-36897 OplogReader.hasNext can return false -> true, confusing checkOplogs
  • SERVER-36906 Connection state now allowed to be kConnectionStateUnknown
  • SERVER-36919 Add server setParameter tlsWithholdClientCertificate (bool)
  • SERVER-36942 Differentiate invalid hostname from invalid certificate
  • SERVER-36947 Enable test commands for perf micro benchmarks
  • SERVER-36980 Remove old aggregation fuzzer from evergreen
  • SERVER-36987 ChunkVersion::minorVersion truncates to 16 bit
  • SERVER-37013 mongoebench-compatible JSON config files attempt to read from views without using “find” command
  • SERVER-37041 Update eval blacklists on older branches to account for differences in tests across branches
  • SERVER-37042 Handle exceptions from cursor.next in ReplSetTest
  • SERVER-37071 Blacklist set7.js and max_doc_size.js from retryable_writes_jscore_stepdown_passthrough
  • SERVER-37127 Update baseline comparison for sys-perf
  • SERVER-37149 Remove deadlock in ScheduleBeforeStartupTest
  • SERVER-37156 benchRun should wait for the worker threads it spawns to exit
  • SERVER-37170 mongos fails to start with error ‘DuplicateKey SetWiredTigerCustomizationHooks’ on dynamic builds
  • SERVER-37216 Android Multiarch builder can’t publish to mavenLocal
  • SERVER-37256 Add missing index_access_methods dependency to storage_wiredtiger_recovery_unit_test
  • TOOLS-1989 Switch evergreen intensive testing to match server priority buildversions
  • TOOLS-2050 oplog_rename_test fails on Windows due to access violation error
  • TOOLS-2099 Tools jstests failing on replica set shutdown
  • WT-3735 Add a workgen workload that generates a lot of page splits
  • WT-3736 Add statistics to measure contention on lookaside cursor
  • WT-3879 Disallow checkpoint from evicting metadata pages
  • WT-3894 Timestamp queue implementation and statistics improvements
  • WT-4090 Low priority reads
  • WT-4104 Fix test/format failure during comparing data content with berkeley db
  • WT-4119 Avoid restarts updating / removing during a column store scan
  • WT-4131 Rename lookaside to cache overflow
  • WT-4144 Fix rollback_to_stable with lookaside history
  • WT-4154 Surface the oldest read timestamp
  • WT-4156 Add new wiredtiger_salvage top level API
  • WT-4176 Expose a WT_SESSION.query_timestamp method
  • WT-4177 Backup cursor open should force a log file switch
  • WT-4185 Don’t remove all lookaside entries when reading a page
  • WT-4211 Add automated test for long running prepared transactions
  • WT-4212 Update lookaside schema to handle prepared transactions
  • WT-4216 Use separate counters for page_swap yield and sleep
  • WT-4218 Change eviction to evict prepared updates
  • WT-4225 Automate a backup test that simulates volume snapshot via dd
  • WT-4231 Fix ctags index of functions with attributes
  • WT-4233 Change log corruption errors to warnings and truncate log
  • WT-4239 Don’t allow checkpoints to perform insert-splits in the tree
  • WT-4241 GNU-stack section should never be conditionally compiled out
  • WT-4243 Fix lookaside sweep to not remove required entries
  • WT-4246 Change transaction update list to support indirect references
  • WT-4248 Fix checkpoints in schema_abort for slow machines
  • WT-4249 Attempt to discard dirty page during verify operation
  • WT-4251 Prepared updates cannot be discarded
  • WT-4252 Btree debug functions can leak scratch buffers on error.
  • WT-4253 Btree debug function to do blind reads doesn’t handle row-store internal pages
  • WT-4256 Loosen check during rollback_to_stable
  • WT-4257 Don’t assume timestamps from lookaside are aligned in memory
  • WT-4259 Restore ref to the previous state rather than MEM when eviction fails
  • WT-4261 Test salvage of out-of-sync metadata/turtle files
  • WT-4262 Lock deleted children in eviction of internal pages
  • WT-4263 Use the right tree when copying a key for a lookaside write
  • WT-4264 Compaction can race with page modifications
  • WT-4267 fixed-length column store operations can corrupt data
  • WT-4268 Random abort should wait until record files exist before starting timer
  • WT-4270 Add an operation field to know where threads hang
  • WT-4272 Increase startup timeout to 30 seconds for slow I/O systems
  • WT-4274 Fix memory leak in wt4156_metadata_salvage test
  • WT-4277 Make truncate in column stores more efficient
  • WT-4281 Shorten runtime of Python test suite
  • WT-4282 Don’t transition pages from limbo to mem unless required
  • WT-4283 Restore WT_ERROR and use a corrupt flag
  • WT-4284 Print a verbose message in recovery on error too
  • WT-4285 Fix wt4156_metadata_salvage Coverity/lint complaints
  • WT-4286 Column store should skip end-of-table checks if there’s an exact match
  • WT-4288 Don’t let return value of closing conn overwrite WT_TRY_SALVAGE
  • WT-4289 Update WT_DATA_CORRUPTION to WT_TRY_SALVAGE in test_txn19.py
  • WT-4291 Fix test_txn19.py error detection by looking for WT_ERROR
  • WT-4292 Add call to testutil_cleanup to avoid memory leak
  • WT-4300 Setting the update timestamp can overwrite the WT_REF.addr field
  • WT-4301 WT_CURSOR.reserve operations can leak memory when committed
  • WT-4305 Add a gating variable for long running prepare support
  • WT-4306 Fix mode if metadata pages need eviction
  • WT-4308 Insert split during sync should not free blocks
  • WT-4321 Disable the random direct I/O test
  • WT-4325 Add a WiredTiger-local version of the qsort(3) call

4.0.2 Changelog

Sharding

  • SERVER-14394 Create initial hashed shard key chunks directly on shards
  • SERVER-25333 Clean up MigrationDestinationManager
  • SERVER-27725 Use batch insert when migrating chunks
  • SERVER-33417 Replace custom majority write catchup with Replication-designed write concern method in the Migration Destination Manager
  • SERVER-35092 ShardServerCatalogCacheLoader should have a timeout waiting for read concern
  • SERVER-35238 Drop the collection after removing the partially written chunks for mapReduce in mongos_manual_intervention_actions.js
  • SERVER-35441 drop/dropDatabase does not cleanup config.tags
  • SERVER-35658 session migration is too sensitive to replica set primary elections
  • SERVER-35676 numInitialChunks Argument is Not Validated
  • SERVER-35720 Allow zones to be created on non-existent collection
  • SERVER-35722 Create internal _shardsvrShardCollection command that runs on the primary shard
  • SERVER-35723 Make the collection critical section into a RAII class
  • SERVER-35794 Read zone information for a collection during internal shardCollection command
  • SERVER-35849 Remove dependency of the write commands on sharding_runtime_d
  • SERVER-36031 Move logic to create initial chunks and update metadata during shard collection to the primary shard
  • SERVER-36071 Check shardsvrShardCollection response status
  • SERVER-36092 Create internal command to create collection on new shard with existing uuid
  • SERVER-36102 Create initial chunks on appropriate shards for zoned sharding
  • SERVER-36322 NamespaceSerializer lock should be used for dropCollection
  • SERVER-36463 Bypass validation of a dummy signatures for isMaster on the unauthenticated connections
  • SERVER-36550 Blacklist drop_sharded_db_tags_cleanup.js in sharding_last_stable_mongos_and_mixed_shards suite
  • SERVER-36849 Disable new shard collection path on v4.0

Replication

  • SERVER-32148 make NamespaceNotFound an acceptable error for emptyCapped and convertToCapped
  • SERVER-32907 Turn heartbeat log verbosity back down in tests
  • SERVER-33243 Improve logging when a node changes its sync source
  • SERVER-33248 Allow choosing a sync source that we are up to date with if it has a higher lastOpCommitted
  • SERVER-35058 Don’t only rely on heartbeat to signal secondary positions in stepdown command
  • SERVER-35126 Create TxnStats class and store it on the Session
  • SERVER-35129 Create a ServerTransactionsMetrics class and store it as a decoration on the ServiceContext
  • SERVER-35146 Track total number of started transactions
  • SERVER-35147 Track total number of committed and aborted transactions in ServerTransactionsMetrics
  • SERVER-35149 Track total number of open transactions in ServerTransactionsMetrics
  • SERVER-35151 Track total number of active and inactive transactions in ServerTransactionsMetrics
  • SERVER-35168 Track information about last client to run a transaction operation on session
  • SERVER-35173 Add autocommit value to transaction sub-document of currentOp
  • SERVER-35174 Add readConcern and readTimestamp to transaction sub-document of currentOp
  • SERVER-35239 AwaitData cursor must handle getMore from client with higher lastKnownCommittedOpTime
  • SERVER-35246 Ignore NamespaceNotFound errors when running collMod during checkReplicaSet in replsettest.js
  • SERVER-35292 Add total number of started transactions to serverStatus
  • SERVER-35293 Add counts of active and inactive transactions to serverStatus
  • SERVER-35294 Add total number of open transactions to serverStatus
  • SERVER-35295 Add total number of committed and aborted transactions to serverStatus
  • SERVER-35300 Track total transaction duration in TxnStats
  • SERVER-35302 Add startWallClockTime to the transaction sub-document of currentOp
  • SERVER-35305 Add timeOpenMicros to the transaction sub-document of currentOp
  • SERVER-35308 Track total active time of transaction in SingleTransactionStats
  • SERVER-35310 Add timeActiveMicros and timeInactiveMicros to transaction sub-document of currentOp
  • SERVER-35388 Improve misleading error messages for aggregation stages banned in transactions
  • SERVER-35428 Add the ability to add two OpDebug objects together
  • SERVER-35432 Add a method for printing out information about a slow transaction
  • SERVER-35433 Log slow transactions when they finish
  • SERVER-35434 Track aggregate OpDebug stats in SingleTransactionStats
  • SERVER-35442 stepdown global lock acqusition should use wait time, not freeze time
  • SERVER-35450 Add information about last client to run transaction operation to currentOp for inactive transactions
  • SERVER-35492 Rename TxnStats to SingleTransactionStats
  • SERVER-35623 Send a replSetStepUp command to an eligible candidate on stepdown
  • SERVER-35624 Enable election handoff by default and update affected tests
  • SERVER-35695 Enable transactions on inMemory storage engine with a command line flag
  • SERVER-35742 Update ServerTransactionsMetrics for failed commits
  • SERVER-35754 Avoid network errors in stopSet() in catchup_takeover_one_high_priority.js
  • SERVER-35766 Replication commands sent in candidate’s new term can interrupt concurrent vote request
  • SERVER-35770 Running a multi-statement transaction when all WiredTiger write tickets are exhausted may lead to deadlock
  • SERVER-35951 Under PV1, ReplicationCoordinatorImpl::processReplSetFreeze for a single node RS should start an election instead of auto-winning.
  • SERVER-35962 buildindexes_false_with_system_indexes.js restarts nodes and should be tagged [requires_persistence]
  • SERVER-36128 ReplicationCoordinatorImpl::fillIsMasterForReplSet should return isMaster:false while in shutdown
  • SERVER-36225 Do not hold replication mutex when calling getMinValid
  • SERVER-36234 Fix ./jstests/replsets/initial_sync_drop_collection.js to prevent using stale heartbeat info in the test.
  • SERVER-36327 Use stepUp command instead of priorities as a way to designate primary nodes in tags.js
  • SERVER-36331 Kill running op when a transaction expires
  • SERVER-36461 Add ‘transaction’ identifier to slow transaction log output
  • SERVER-36592 Blacklist snapshot_read_kill_op_only.js workload from backup_restore.js lib
  • SERVER-36664 Use awaitReplication in read_committed_with_catalog_changes.js instead of getLastError
  • SERVER-36685 Reverse order of TransactionParticipant (Session in 4.0) and Client locks

Query

  • SERVER-26387 Replace noPassthrough/indexbg2.js with test that uses failpoints
  • SERVER-34846 Covered index with collated field returns incorrect result when collation not involved in match or sort
  • SERVER-36239 MatchExpression parser query rule

Aggregation

  • SERVER-36070 Aggregation with $out results in error when Auditing is enabled
  • SERVER-36123 Reject $out with mode: “replaceCollection” if the output collection is sharded
  • SERVER-36715 Pushing $sort stage to query system can leave pipeline in an unstitched state

JavaScript

SERVER-35986 Stop running eval command in parallel suite on older branches

Storage

  • SERVER-32994 Create a script to rebuild unique indexes in right format after a downgrade
  • SERVER-33740 Add Evergreen task for running powercycle against mobile storage engine
  • SERVER-35367 Hold locks in fewer callers of waitForAllEarlierOplogWritesToBeVisible()
  • SERVER-35386 Wait for the secondary’s majority commit point to advance before pausing batch application in secondary_reads_timestamp_visibility.js
  • SERVER-35473 Mobile SE: Fix writeConflictRetry loop with map-reduce jstests
  • SERVER-35845 Add tag to noPassthrough/indexbg2.js
  • SERVER-36005 IndexCatalogEntryImpl::_catalogIsReady is unsafe to unconditionally call.
  • SERVER-36062 Mobile SE: Stop running parallel and concurrent suites on mobile variants
  • SERVER-36238 replica set startup fails in wt_cache_full.js, initial_sync_wt_cache_full.js, recovery_wt_cache_full.js when journaling is disabled
  • SERVER-36397 Embedded Server SDK Should Return Error When TTL Index Creation is Attempted

WiredTiger

  • SERVER-28734 Recover WiredTiger data files when lacking metadata, but have _mdb_catalog data

Operations

  • SERVER-34664 Commands that are unsupported in a transaction should error
  • SERVER-35180 Safeguard from setting operation session info values while in a direct client
  • SERVER-35617 Unpleasant db.enableFreeMonitoring “not master” error on newly launched mongod
  • SERVER-36294 Sum of currentActive and currentInactive not equal to currentOpen transactions
  • SERVER-36479 Log redaction does not show planSummary when slow queries are logged

Build and Packaging

Internals

  • SERVER-32281 Thread pool task executor ignores errors returned from startCommand()
  • SERVER-33695 Include the loop name in the before and after recovery files in powertest.py
  • SERVER-34258 Error from mount_drives.sh on Windows
  • SERVER-34558 Add SSL_version to client metadata logging
  • SERVER-35003 Consider making the balancer threshold to always be 1
  • SERVER-35025 Tests depending on exact times removeShard is called are invalid
  • SERVER-35037 Create new concurrency suite that runs consecutive operations inside of a single transaction
  • SERVER-35056 Flush ready callbacks on NetworkInterfaceTL shutdown
  • SERVER-35100 Do not log a Python stack trace when a hook dynamic test fails
  • SERVER-35110 Log locations in set_feature_compatibility_version.js where fatal assertions/other errors are expected
  • SERVER-35215 Future::onError<ErrorCodes::Error>() to let you register a handler for a single code.
  • SERVER-35234 makePromiseFuture<T>() to create a bound Promise and Future
  • SERVER-35244 Dropping config db in 4.0.0-rc0 fails while it passes in 3.7.9 and earlier
  • SERVER-35537 Create version of benchRun() which can be used with embedded
  • SERVER-35559 Update transaction retry functions to not call abort after commit
  • SERVER-35592 Create Java bindings for the Embedded CAPI
  • SERVER-35654 rollback_transaction_table.js fails even in WT runs
  • SERVER-35661 Add 5 second syncdelay to rollback shutdown fuzzers
  • SERVER-35684 Remove promise.getFuture()
  • SERVER-35706 Fix race in ctor for LogicalSessionCacheImpl
  • SERVER-35775 Unhelpful Message String in db.getFreeMonitoringStatus()
  • SERVER-35919 Ensure all tests that “use transactions” perform collection drops with w:majority
  • SERVER-35946 Powercycle kill_mongod function should ensure the service is not in a running state
  • SERVER-36001 Add requires_document_locking tag to noPassthrough/indexbg2.js
  • SERVER-36067 Upload artifacts from running install-mobile-test target in Evergreen to S3
  • SERVER-36069 Vendor mongoebench-compatible JSON config files from mongodb/mongo-perf into src/third_party
  • SERVER-36073 Save stats from BenchRunner::finish() to a JSON file in mongoebench
  • SERVER-36129 Concurrency stepdown suites should wait for replication of workload setups before starting stepdown thread
  • SERVER-36191 Cleanup logic for converting BSONElement to strings
  • SERVER-36245 Create a multi-arch android variant
  • SERVER-36274 Re-enable –ycsb-throughput-analysis in sys-perf
  • SERVER-36301 build WT with HAVE_NO_CRC32_HARDWARE on RHEL 6.7 s390x
  • SERVER-36399 Fill in tasks for Cloud nightly tests
  • SERVER-36406 db._authOrThrow should prioritize user specified authenticationMechanism instead of server’s mechanisms
  • SERVER-36448 Disable election handoff in suites that use the ContinuousStepdown hook
  • SERVER-36462 Add atlas user to runtime_secret.yml
  • SERVER-36466 Secure shutdown conditions for SpecificPool
  • SERVER-36623 Do not rebuild indexes before repairing databases
  • WT-3276 Add recover=salvage to recover from a corrupted log file
  • WT-3856 Create a test that runs recovery to different points of time with schema operations
  • WT-3943 Include full error message when a python test asserts
  • WT-3955 Add verbose option to log more messages on error returns
  • WT-3963 Add a schema intensive abort testing
  • WT-3968 Use compression ratio to tune page sizes
  • WT-4010 Simplify test/format timestamp handling.
  • WT-4026 Add implementation for existing file extension configuration API
  • WT-4134 Rework assertion that we don’t discard required history
  • WT-4147 Log recovery should not ignore corruption outside of log records in a log file
  • WT-4160 Restore performance when timestamps are not in use
  • WT-4168 Update upgrading documentation for 3.1.0 release
  • WT-4169 Fix wt verify dump-pages failure
  • WT-4171 Enabling tree walk timing stress causes excessive slowdown
  • WT-4172 Add diagnostic hazard pointer checks in more places before freeing refs
  • WT-4174 Do not access the lookaside file in rollback_to_stable when running with in_memory=true
  • WT-4178 Fixes for wt_btree_immediately_durable needed for in-memory
  • WT-4179 Expose WiredTiger crc32c functions
  • WT-4182 Use conservative approach for log checksum errors
  • WT-4183 Extend verbose option to log more messages on error returns
  • WT-4186 Log recovery should detect and report corruption within log records
  • WT-4187 Coverity: unused value complaints
  • WT-4188 Coverity: unchecked return value complaints
  • WT-4189 Potential infinite loop in __async_flush_wait().
  • WT-4191 Fix Coverity static analysis errors
  • WT-4193 test/format snapshot-isolation search mismatch
  • WT-4194 Improve fairness of eviction with multiple tables
  • WT-4195 When encountering an illegal value, log the value that failed
  • WT-4196 Make log corruption checking work regardless of the machine byte order
  • WT-4198 Some supported MongoDB architectures don’t support crc32 hardware
  • WT-4199 Fix an incorrect report of log corruption
  • WT-4201 Fix Coverity static analysis issues
  • WT-4206 Fix error handling in cursor close routines
  • WT-4207 Coverity #1394567: null pointer dereference
  • WT-4208 tree walks can be interrupted by locked internal pages
  • WT-4210 schema abort child process failing prematurely
  • WT-4213 Rename lock statistics that have redundant or misleading text
  • WT-4215 Allow recovery of backup without salvage
  • WT-4226 test/format LSM configurations can misconfigure prepare and timestamps
  • WT-4229 Lint
  • WT-4234 Remove documentation mention of legacy tool statlog.py
  • WT-4235 Fix workgen tracking of table state across workloads
  • WT-4242 New log file extension Python test failure

4.0.1 Changelog

Security

  • SERVER-35125 null pointer read access violation in SSLHandshakeManager::doServerHandshake
  • SERVER-36027 Enterprise build does not recognize the –redactClientLogData flag

Sharding

  • SERVER-33237 Optimize the Range Deleter speed
  • SERVER-33697 Provide sanity check on number of cached sessions
  • SERVER-34897 Introduce parameter to control whether MongoS should automatically retry failed find commands
  • SERVER-35609 Create initial chunk on primaryShard if only writing one chunk
  • SERVER-35632 Blacklist lagged_config_secondary.js and all_config_servers_blackholed_from_mongos.js from RHEL 6.7 s390x variants
  • SERVER-35691 Make all code paths returning StaleConfig error go through the StaleConfigInfo serializer
  • SERVER-35711 Remove unused anonymous function getPersistedMaxDbVersion() from shard server catalog cache loader
  • SERVER-35745 _getNextSessionMods doesn’t include oplogReplay flag to query new oplog created during migration
  • SERVER-35938 NamespaceSerializer lock should be used for database as well as collection during createCollection
  • SERVER-35997 Await replication on config server after shardCollection in safe secondary reads tests
  • SERVER-36041 Increase the size of changelog and actionlog
  • SERVER-36075 Add comment to shard_identity_rollback.js explaining why we set the FCV to 4.0 on the shardsvr
  • SERVER-36132 Invariant that chunk migration is actually reflected in the metadata after successful commit
  • SERVER-36232 Refresh after chunk migration commit may not see the committed metadata

Replication

  • SERVER-32088 ChangeStream resumeAfter does not work on sharded collections if not all shards have chunks for the collection
  • SERVER-34414 Creating a role with buildsIndexes:false node hits an fassert
  • SERVER-34758 replSetGetStatus can deadlock with initialSyncer
  • SERVER-35124 Stepdown suites with MMAP V1 often fail due to flushing mmaps taking long time
  • SERVER-35200 Speed up failure detection in the OplogFetcher during steady state replication
  • SERVER-35388 Improve misleading error messages for aggregation stages banned in transactions
  • SERVER-35488 ReplSetTest.waitForState() should ensure node has finished closing connections
  • SERVER-35571 Wait until all nodes become stable before checkOplogs
  • SERVER-35991 Turn off chaining in set_feature_compatibility_version.js
  • SERVER-35992 Secondary should have no vote in SecondaryReadsTest

Query

  • SERVER-34789 Using resume token from an ‘invalidate’ notification with ‘resumeAfter’ should error
  • SERVER-34933 pcre verb support
  • SERVER-35693 Parsing of $in takes quadratic time due to O(n^2) boost::flat_set constructor
  • SERVER-35751 Make kill_own_ops.js more robust
  • SERVER-35851 Consider checking whether the array passed to $in is already sorted before calling std::sort
  • SERVER-35929 Possible use-after-free when reloading the view catalog due to an invalidation

Aggregation

  • SERVER-35028 Add change stream notifications for collection drop and rename
  • SERVER-35029 Add change stream notification for database drop
  • SERVER-35084 change_stream_enforce_max_time_ms_on_mongos.js expects getMore to schedule follow-up getMores
  • SERVER-35634 view_catalog_cycle_lookup.js should not always assert that finds on views will succeed
  • SERVER-35961 Remove uninitialized count variable in MapReduce command

Catalog

SERVER-35563 The UUIDCatalog onCreateCollection observer should atomically unregister and re-register catalog entries

Storage

  • SERVER-32509 Mobile SE: Implement error reporting for unsupported startup options
  • SERVER-32997 Mobile SE: Design and implement multi-reader or single-writer concurrency
  • SERVER-33605 Mobile SE: Disable capped collections
  • SERVER-33651 Mobile SE: Use full synchronous mode for SQLite writes
  • SERVER-34002 readConcern_snapshot.js should wait for writes to be majority-committed on secondaries before reading
  • SERVER-34113 Remove all support for snapshot reads outside of multi-document transactions
  • SERVER-34129 Don’t hold database or collection locks during oplog truncation
  • SERVER-34579 Do not populate indexDetails for mobile storage engine
  • SERVER-34713 Progressively declining dropDatabase performance
  • SERVER-35085 repair can cause spurious NamespaceNotFound errors with concurrent initial sync operations
  • SERVER-35317 restartCatalog problematically discards minVisibleSnapshot data
  • SERVER-35398 Mobile SE: Remove code for capped collection
  • SERVER-35671 DatabaseHolderImpl::closeAll can leave catalog in an incomplete state
  • SERVER-35704 Tag readConcern_snapshot.js with ‘uses_transactions’
  • SERVER-35789 check for index catalog isready mismatch for multi-doc txns
  • SERVER-35859 Disable rocksdb buildvariant
  • SERVER-35994 Reduce CPU load in secondary_reads_unique_indexes.js test
  • SERVER-36006 multiInitialSyncApply should consider setting a read timestamp of kNoTimestamp
  • SERVER-36025 RestartCatalogCommand can attempt to reinitialize oplog pointers on standalones
  • SERVER-36167 Add pretty printer for WT flags members

GridFS

SERVER-35361 filemd5 command fails to safely clean up PlanExecutor after manual yield

Operations

  • SERVER-27264 With net.ssl.allowConnectionsWithoutCertificates: true, allow disabling no client certificate log warnings
  • SERVER-32064 A logical session id should be included in all command requests from the mongo shell
  • SERVER-34160 Mongo client run buffered command when terminated.
  • SERVER-35758 Mongo shell prompt errors when running transactions after overriding “db”
  • SERVER-35903 Free monitoring doesn’t allow disabling from the shell without enabling it first
  • SERVER-36010 Change log messages for Windows stacktraces to use error() or severe() rather than log()
  • SERVER-36088 Replica set connection strings trigger access violation on 4.0 shell + Windows

Build and Packaging

Tools

  • TOOLS-2058 mongoreplay does not show OP_MSG commands
  • TOOLS-2062 Support zlib compression in mongoreplay
  • TOOLS-2075 mongoreplay always replays to secondary

Internals

  • SERVER-33817 Powercycle test using kill mongod
  • SERVER-34563 Connect via SRV record fails if the returned record resides in subdomain
  • SERVER-34793 Add call to BF suggestion server on failed task completion
  • SERVER-34810 Session cache refresh can erroneously kill cursors that are still in use
  • SERVER-34956 big_object1.js is not resilient to unexpected stepdowns
  • SERVER-34984 Update major_version_upgrade.js test to call setFCV to the latest FCV
  • SERVER-34996 Save console_output & console_screenshot from aws_ec2.py as artifacts in evergreen.yml
  • SERVER-35101 Handle differently a badly formed bindIp argument
  • SERVER-35165 Disable and re-enable update_test_lifecycle Evergreen task on the 4.0 branch
  • SERVER-35188 Typo in ServiceLiason* types
  • SERVER-35263 Add FSM workloads for testing atomicity and isolation of updates inside a transaction across multiple collections and databases
  • SERVER-35312 Update system_perf.yml and perf.yml to compare to 3.6.5 baseline for master and 4.0
  • SERVER-35313 CleanupConcurrencyWorkloads resmoke hook needs to handle the balancer
  • SERVER-35383 Increase electionTimeoutMillis for the ContinuousStepdown hook used in stepdown suites
  • SERVER-35389 Remove dead code from old FSM suite
  • SERVER-35506 The Powercycle wait_for_mongod_shutdown function should ensure the mongod process is no longer running
  • SERVER-35523 FSMWorkloadTestCase erroneously sets TestData.sameDB=true and TestData.sameCollection=true
  • SERVER-35550 Bridge should not block listener while making outbound connections
  • SERVER-35578 Switch package tests to new package testing VPC
  • SERVER-35588 powertest.py should call replSetReconfigure command only after successful replSetGetConfig
  • SERVER-35627 Repair should re-create missing collection data files from the existing metadata
  • SERVER-35664 Run the android embedded tests on api24 system images
  • SERVER-35668 Avoid moving concurrency tests to small instances on arm64
  • SERVER-35675 find_by_uuid_and_rename.js should not fail from QueryPlanKilled when yielding during a collection rename
  • SERVER-35692 Add support to an optional re-registration bool in metrics response
  • SERVER-35702 Stop running the agg and update fuzzer on macOS
  • SERVER-35724 Remote EC2 hosts which are not accessible via ssh should fail with system error
  • SERVER-35727 make the embedded sdk compile task in to a multi-task task group.
  • SERVER-35834 Fix compile errors in dns_name_test on Microsoft compilers
  • SERVER-35850 Update filename suffix to v4.0-latest for nightly builds
  • SERVER-35858 Error in call BF Suggestion service
  • SERVER-35908 Shutdown service entry point in mongoed before shutting down embedded
  • SERVER-35990 Update evergreen_task_timeout.py REQUIRED_BUILD_VARIANTS list
  • SERVER-35993 read_concern_uninitiated_set restarts nodes so must not allow ephemeral storage
  • SERVER-36055 Mobile SE: Compact returns wrong error code on mobile
  • SERVER-36144 want to add an empty cloud_nightly.yml file
  • TOOLS-1991 Build tools with Go 1.10.1
  • WT-3839 Document the undefined behavior when a range truncate overlaps with inserts
  • WT-3917 Enhance WT_CURSOR::reserve documentation around commit visibility
  • WT-4024 Fix a race between split and next/prev
  • WT-4048 Generalize timing_stress_for_test split functionality
  • WT-4067 Enhance LSM to not pin as much history in cache
  • WT-4101 Don’t abort the eviction server during session verify when oldest_timestamp is held back
  • WT-4111 Improve checkpoint scrubbing algorithm
  • WT-4125 Ensure that subsequent checkpoints with stable timestamp don’t read too much
  • WT-4133 Coverity 1393445, 1393446 Dereference before null check
  • WT-4136 Add a new timing stress flag that yields during tree search
  • WT-4138 Add an option to timeout waiting for space in the cache
  • WT-4139 rename the cursor restart statistic to match implementation
  • WT-4140 Cursor walk limits quick eviction page selection unnecessarily.
  • WT-4141 Enhance checkpoint with timestamps to unblock eviction sooner
  • WT-4143 Use WiredTiger.turtle.set if it exists but WiredTiger.turtle does not
  • WT-4145 Only include the checkpoint timestamp during checkpoints
  • WT-4146 Coverity 1393639, unused variable
  • WT-4152 Save return value for later comparison in transaction code
  • WT-4163 Lint