Docs Menu

Docs HomeDevelop ApplicationsMongoDB Manual

Database Profiler

On this page

  • Profiling Levels
  • Enable and Configure Database Profiling
  • View Profiler Data
  • Profiler Overhead

The database profiler collects detailed information about Database Commands executed against a running mongod instance. This includes CRUD operations as well as configuration and administration commands.

The profiler writes all the data it collects to a system.profile collection, a capped collection in each profiled database. See Database Profiler Output for an overview of the system.profile documents created by the profiler.

The profiler is off by default. You can enable the profiler on a per-database or per-instance basis at one of several profiling levels.

When enabled, profiling has an effect on database performance and disk use. See Database Profiler Overhead for more information.

This page shows important administration options for the database profiler. For additional information, see:

Warning

Do not attempt to create a time series collection or view with the name system.profile. MongoDB 6.3 and later versions return an IllegalOperation error if you attempt to do so. Earlier MongoDB versions crash.

The following profiling levels are available:

Level
Description
0
The profiler is off and does not collect any data. This is the default profiler level.
1

The profiler collects data for operations that take longer than the value of slowms or that match a filter.

When a filter is set:

  • The slowms and sampleRate options are not used for profiling.

  • The profiler only captures operations that match the filter.

2
The profiler collects data for all operations.

You can enable database profiling for mongod instances.

This section shows how you use the mongosh helper method db.setProfilingLevel() to enable profiling. To use a driver method instead, see the driver documentation.

To enable profiling for a mongod instance, set the profiling level to a value greater than 0. The profiler records data in the system.profile collection. MongoDB creates the system.profile collection in a database after you enable profiling for that database.

To enable profiling and set the profiling level, pass the profiling level to the db.setProfilingLevel() helper. For example, to enable profiling for all database operations for the currently connected database, run this operation in mongosh:

db.setProfilingLevel(2)

The shell returns the previous profiling level in the was field and sets the new level. In the following output, the "ok" : 1 key-value pair indicates the operation succeeded:

{ "was" : 0, "slowms" : 100, "sampleRate" : 1.0, "ok" : 1 }

To verify the new setting, see the Check Profiling Level section.

Starting in MongoDB 5.0, changes made to the database profiler level, slowms, sampleRate, or filter using the profile command or db.setProfilingLevel() wrapper method are recorded in the log file.

The slowms and sampleRate profiling settings are global. When set, these settings affect all databases in your process.

When set through the profile command or db.setProfilingLevel() shell helper method, profiling level and filter settings are set at the database level. When set as either command line or configuration file options, profiling level and filter settings affect the entire process.

By default, the slow operation threshold is 100 milliseconds. To change the slow operation threshold, specify the required threshold value in one of the following ways:

The following example sets the profiling level for the currently connected database to 1 and sets the slow operation threshold for the mongod instance to 20 milliseconds:

db.setProfilingLevel( 1, { slowms: 20 } )

A profiling level of 1 causes the profiler to record operations slower than the slowms threshold.

Important

The slow operation threshold applies to all databases in a mongod instance. It is used by both the database profiler and the diagnostic log and should be set to the highest useful value to avoid performance degradation.

You can use db.setProfilingLevel() to configure slowms and sampleRate for mongos. For the mongos, the slowms and sampleRate configuration settings only affect the diagnostic log and not the profiler since profiling is not available on mongos. [1]

The following example sets a mongos instance's slow operation threshold for logging slow operations to 20:

db.setProfilingLevel( 0, { slowms: 20 } )

Starting in MongoDB 4.2, the profiler entries and the diagnostic log messages (i.e. mongod/mongos log messages) for read/write operations include:

  • queryHash to help identify slow queries with the same query shape.

  • planCacheKey to provide more insight into the query plan cache for slow queries.

Starting in version 4.2, secondary members of a replica set now log oplog entries that take longer than the slow operation threshold to apply. These slow oplog messages:

  • Are logged for the secondaries in the diagnostic log.

  • Are logged under the REPL component with the text applied op: <oplog entry> took <num>ms.

  • Do not depend on the log levels (either at the system or component level)

  • Do not depend on the profiling level.

  • Are affected by slowOpSampleRate.

The profiler does not capture slow oplog entries.

To profile only a randomly sampled subset of all slow operations, specify the desired sample rate in one of the following ways: [2]

By default, sampleRate is set to 1.0, meaning all slow operations are profiled. When sampleRate is set between 0 and 1, databases with a profiling level 1 only profile a randomly sampled percentage of slow operations based on sampleRate.

The following example sets the profiling level for the currently connected database to 1 and sets the profiler to sample 42% of all slow operations:

db.setProfilingLevel( 1, { sampleRate: 0.42 } )

The modified sample rate value also applies to the system log.

You can use db.setProfilingLevel() to configure slowms and sampleRate for mongos. For the mongos, the slowms and sampleRate configuration settings only affect the diagnostic log and not the profiler because profiling isn't available on mongos. [1]

For example, the following sets a mongos instance's sampling rate for logging slow operations:

db.setProfilingLevel( 0, { sampleRate: 0.42 } )

Important

When logLevel is set to 0, MongoDB records slow operations to the diagnostic log at a rate determined by slowOpSampleRate.

At higher logLevel settings, all operations appear in the diagnostic log regardless of their latency with the following exception: the logging of slow oplog entry messages by the secondaries. The secondaries log only the slow oplog entries; increasing the logLevel does not log all oplog entries.

[1](1, 2) See Database Profiling and Sharding.

You can set a filter to control which operations are profiled and logged. You can set the profiling filter in one of the following ways:

For mongod instances, the filter affects both the diagnostic log and, if enabled, the profiler.

For mongos instances, the filter affects the diagnostic log only and not the profiler since profiling is not available on mongos.

Note

When a profiling filter is set, the slowms and sampleRate options do not affect the diagnostic log or the profiler.

The following db.setProfilingLevel() example sets the profile level for the currently connected database:

  • the profiling level to 2,

  • the filter of { op: "query", millis: { $gt: 2000 } }, which causes the profiler to only log query operations that took longer than 2 seconds.

db.setProfilingLevel( 2, { filter: { op: "query", millis: { $gt: 2000 } } } )

To view the profiling level, run the following example in mongosh:

db.getProfilingStatus()

The shell returns a document similar to the following:

{ "was" : 0, "slowms" : 100, "sampleRate" : 1.0, "ok" : 1 }

The was field indicates the current profiling level.

The slowms field indicates operation time threshold, in milliseconds, beyond which operations are considered slow.

The sampleRate field indicates the percentage of slow operations that should be profiled.

To disable profiling, run the following example in mongosh:

db.setProfilingLevel(0)

Note

Disabling profiling can improve database performance and lower disk use. For more information, see Database Profiler Overhead .

For development and test environments, you can enable database profiling for an entire mongod instance. The profiling level applies to all databases provided by the mongod instance.

To enable profiling for a mongod instance, pass the following options to mongod at startup.

mongod --profile 1 --slowms 15 --slowOpSampleRate 0.5

Alternatively, you can specify operationProfiling in the configuration file.

This sets the profiling level to 1, defines slow operations as those that last longer than 15 milliseconds, and specifies that only 50% of slow operations should be profiled. [2]

The slowms and slowOpSampleRate also affect the operations that are recorded in the diagnostic log when logLevel is set to 0. The slowms and slowOpSampleRate are also available to configure diagnostic logging for mongos. [2]

Tip

See also:

You cannot enable profiling on a mongos instance. To enable profiling in a sharded cluster, you must enable profiling for each mongod instance in the cluster.

However, starting in MongoDB 4.0, you can set the --slowms and slowOpSampleRate on mongos to configure the diagnostic log for slow operations.

The database profiler logs information about database operations in the system.profile collection.

To view profiling information, query the system.profile collection. To view example queries, see Example Profiler Data Queries. For an explanation of the output data, see Database Profiler Output.

It is no longer possible to perform any operation, including reads, on the system.profile collection from within a transaction.

Tip

You can use $comment to add data to the query predicate to make it easier to analyze data from the profiler.

This section shows example queries on the system.profile collection. For query output details, see Database Profiler Output.

To return the most recent 10 log entries in the system.profile collection, run a query similar to the following:

db.system.profile.find().limit(10).sort( { ts : -1 } ).pretty()

To return all operations except command operations ($cmd), run a query similar to the following:

db.system.profile.find( { op: { $ne : 'command' } } ).pretty()

To return operations for a particular collection, run a query similar to the following. This example returns operations in the mydb database's test collection:

db.system.profile.find( { ns : 'mydb.test' } ).pretty()

To return operations that take longer than 5 milliseconds to complete, run:

db.system.profile.find( { millis : { $gt : 5 } } ).pretty()

To return operations for a specific time range, run:

db.system.profile.find( {
ts : {
$gt: new ISODate("2012-12-09T03:00:00Z"),
$lt: new ISODate("2012-12-09T03:40:00Z")
}
} ).pretty()

The following example looks at the time range, suppresses the user field from the output to make it easier to read, and sorts the results by how long each operation took to run:

db.system.profile.find( {
ts : {
$gt: new ISODate("2011-07-12T03:00:00Z"),
$lt: new ISODate("2011-07-12T03:40:00Z")
}
}, { user: 0 } ).sort( { millis: -1 } )

On a database that has profiling enabled, the show profile helper in mongosh displays the 5 most recent operations that took at least 1 millisecond to execute. Run show profile from mongosh:

show profile

When enabled, profiling has an effect on database performance, especially when configured with a profiling level of 2, or when using a low threshold value with a profiling level of 1.

Profiling also uses disk space, because profiling writes logs to the system.profile collection and the MongoDB logfile.

Warning

Consider performance and storage implications before you enable the profiler in a production deployment.

The system.profile collection is a capped collection with a default size of 1 megabyte. A collection of this size can typically store several thousand profile documents, but some applications may use more or less profiling data per operation. If you need to change the size of the system.profile collection, follow the steps below.

To change the size of the system.profile collection on the primary, you must:

  1. Disable profiling.

  2. Drop the system.profile collection.

  3. Create a new system.profile collection.

  4. Re-enable profiling.

For example, to create a new system.profile collection that is 4000000 bytes (4 MB), use the following sequence of operations in mongosh:

db.setProfilingLevel(0)
db.system.profile.drop()
db.createCollection( "system.profile", { capped: true, size:4000000 } )
db.setProfilingLevel(1)

To change the size of the system.profile collection on a secondary, you must stop the secondary, run it as a standalone, and then perform the steps above. When done, restart the standalone as a member of the replica set. For more information, see Perform Maintenance on Replica Set Members.

[2](1, 2, 3) Starting in version 4.2, secondary members of a replica set now log oplog entries that take longer than the slow operation threshold to apply. These slow oplog messages:
  • Are logged for the secondaries in the diagnostic log.
  • Are logged under the REPL component with the text applied op: <oplog entry> took <num>ms.
  • Do not depend on the log levels (either at the system or component level)
  • Do not depend on the profiling level.
  • Are affected by slowOpSampleRate.
The profiler does not capture slow oplog entries.
←  Tuning Your Connection Pool SettingsDatabase Profiler Output →