Queries¶
On this page
Queries¶
Mongoid provides a rich query DSL inspired by ActiveRecord. A trivial query looks as follows:
A more complex query utilizing various Mongoid features could be as follows:
The query methods return Mongoid::Criteria
objects, which are chainable
and lazily evaluated wrappers for MongoDB query language (MQL).
The queries are executed when their result sets are iterated. For example:
Methods like first
and last
return the individual documents immediately.
Otherwise, iterating a Criteria object with methods like each
or map
retrieves the documents from the server. to_a
can be used to force
execution of a query that returns an array of documents, literally converting
a Criteria object to an Array.
When a query method is called on a Criteria instance, the method returns a new Criteria instance with the new conditions added to the existing conditions:
Condition Syntax¶
Mongoid supports three ways of specifying individual conditions:
- Field syntax.
- MQL syntax.
- Symbol operator syntax.
All syntaxes support querying embedded documents using the dot notation. All syntaxes respect field types, if the field being queried is defined in the model class, and field aliases.
The examples in this section use the following model definition:
Field Syntax¶
The simplest querying syntax utilizes the basic Ruby hashes. Keys can be symbols or strings, and correspond to field names in MongoDB documents:
MQL Syntax¶
An MQL operator may be specified on any field using the hash syntax:
Symbol Operator Syntax¶
MQL operators may be specified as methods on symbols for the respective field name, as follows:
Embedded Documents¶
To match values of fields of embedded documents, use the dot notation:
Note
Queries always return top-level model instances, even if all of the conditions are referencing embedded documents.
Field Types¶
In order to query on a field, it is not necessary to add the field to the model class definition. However, if a field is defined in the model class, the type of the field is taken into account when constructing the query:
Aliases¶
Queries take into account storage field names and field aliases:
Since id and _id fields are aliases, either one can be used for queries:
Logical Operations¶
Mongoid supports and
, or
, nor
and not
logical operations on
Criteria
objects. These methods take one or more hash of conditions
or another Criteria
object as their arguments, with not
additionally
having an argument-free version.
For backwards compatibility with earlier Mongoid versions, all of the logical operation methods also accept arrays of parameters, which will be flattened to obtain the criteria. Passing arrays to logical operations is deprecated and may be removed in a future version of Mongoid.
The following calls all produce the same query conditions:
Operator Combinations¶
As of Mongoid 7.1, logical operators (and
, or
, nor
and not
)
have been changed to have the the same semantics as those of ActiveRecord.
To obtain the semantics of or
as it behaved in Mongoid 7.0 and earlier,
use any_of
which is described below.
When conditions are specified on the same field multiple times, all conditions are added to the criteria:
any_of
, nor
and not
behave similarly, with not
producing
different query shapes as described below.
When and
, or
and nor
logical operators are used, they
operate on the criteria built up to that point and its argument.
where
has the same meaning as and
:
and
Behavior¶
The and
method will add new simple conditions to the top level of the
criteria, unless the receiving criteria already has a condition on the
respective fields, in which case the conditions will be combined with $and
.
As of Mongoid 7.1, specifying multiple criteria on the same field with and
combines all criteria so specified, whereas in previous versions of Mongoid
conditions on a field sometimes replaced previously specified conditions on
the same field, depending on which form of and
was used.
or
/nor
Behavior¶
or
and nor
produce $or
and $nor
MongoDB operators, respectively,
using the receiver and all of the arguments as operands. For example:
If the only condition on the receiver is another or
/nor
, the new
conditions are added to the existing list:
Use any_of
to add a disjunction to a Criteria object while maintaining
all of the conditions built up so far as they are.
any_of
Behavior¶
any_of
adds a disjunction built from its arguments to the existing
conditions in the criteria. For example:
The conditions are hoisted to the top level if possible:
not
Behavior¶
not
method can be called without arguments, in which case it will negate
the next condition that is specified. not
can also be called with one
or more hash conditions or Criteria
objects, which will all be negated and
added to the criteria.
Note
$not
in MongoDB server cannot be used with a string argument.
Mongoid uses $ne
operator to achieve such a negation:
Similarly to and
, not
will negate individual conditions for simple
field criteria. For complex conditions and when a field already has a condition
defined on it, since MongoDB server only supports the $not
operator on
a per-field basis rather than globally, Mongoid emulates $not
by using
an {'$and' => [{'$nor' => ...}]}
construct:
If using not
with arrays or regular expressions, please note the
caveats/limitations of $not
stated in the MongoDB server documentation.
Incremental Query Construction¶
By default, when conditions are added to a query, Mongoid considers each
condition complete and independent from any other conditions potentially
present in the query. For example, calling in
twice adds two separate
$in
conditions:
Some operator methods support building the condition incrementally. In this case, when an condition on a field which uses one of the supported operators is being added, if there already is a condition on the same field using the same operator, the operator expressions are combined according to the specified merge strategy.
Merge Strategies¶
Mongoid provides three merge strategies:
- Override: the new operator instance replaces any existing conditions on the same field using the same operator.
- Intersect: if there already is a condition using the same operator on the same field, the values of the existing condition are intersected with the values of the new condition and the result is stored as the operator value.
- Union: if there already is a condition using the same operator on the same field, the values of the new condition are added to the values of the existing condition and the result is stored as the operator value.
The following snippet demonstrates all of the strategies, using in
as the
example operator:
The strategy is requested by calling override
, intersect
or union
on a Criteria
instance. The requested strategy applies to the next
condition method called on the query. If the next condition method called does
not support merge strategies, the strategy is reset, as shown in the following
example:
Since ne
does not support merge strategies, the union
strategy was
ignored and reset and when in
was invoked the second time there was no
strategy active.
Supported Operator Methods¶
The following operator methods support merge strategies:
all
in
nin
The set of methods may be expanded in future releases of Mongoid. For future compatibility, only invoke a strategy method when the next method call is an operator that supports merge strategies.
Note that the merge strategies are currently only applied when conditions are
added through the designated methods. In the following example merge strategy
is not applied because the second condition is added via where
, not via
in
:
This behavior may change in a future release of Mongoid and should not be relied upon.
In contrast, it does not matter how the existing query was built when a
merge strategy-supporting operator method is invoked. In the following
example, the first condition was added through where
but the strategy
mechanism still applies:
Operator Value Expansion¶
Operator methods that support merge strategies all take Array
as their value
type. Mongoid expands Array
-compatible types, such as a Range
,
when they are used with these operator methods:
Additionally, Mongoid has historically wrapped non-Array
values in arrays,
as the following example demonstrates:
This wrapping behavior is deprecated and should not be relied on. It may be removed in a future release of Mongoid.
Query Methods¶
elem_match¶
This matcher finds documents with array fields where one of the array values matches all of the conditions. For example:
elem_match
also works with embedded associations:
elem_match
does not work with non-embedded associations because MongoDB
does not have joins - the conditions would be added to the collection
that is the source of a non-embedded association rather than the collection
of the association’s target.
elem_match
can also be used with recursively embedded associations,
as the following example shows:
Projection¶
only
¶
The only
method retrieves only the specified fields from the database. This
operation is sometimes called “projection”.
Attempting to reference attributes which have not been loaded results in
ActiveModel::MissingAttributeError
.
Even though Mongoid currently allows writing to attributes that have not been loaded, such writes will not be persisted (MONGOID-4701) and should therefore be avoided.
only
can also be used with embedded associations:
Note
Server versions 4.2 and lower allowed projecting both an association and the association’s fields in the same query, as follows:
The most recent projection specification overrides the earlier one. For example, the above query was equivalent to:
Server versions 4.4 and higher prohibit specifying an association and its fields in projection in the same query.
only
can be specified with referenced associations (has_one, has_many,
has_and_belongs_to_many) but is currently ignored for referenced associations -
all fields of referenced associations will be loaded
(MONGOID-4704).
Note that if a document has has_one
or has_and_belongs_to_many
associations,
the fields with foreign keys must be included in the list of attributes
loaded with only
for those associations to be loaded. For example:
without
¶
The opposite of only
, without
causes the specified fields to be omitted:
Because Mongoid requires the _id
field for various operations, it (as well
as its id
alias) cannot be omitted via without
:
Ordering¶
Mongoid provides the order
method on Criteria
objects and its alias,
order_by
, to specify the ordering of documents. These methods take a
hash indicating which fields to order the documents by, and whether to use
ascending or descending order for each field.
The direction may be specified as integers 1
and -1
for ascending
and descending, respectively, or as symbols :asc
and :desc
, or as
strings "asc"
and "desc"
.
Alternatively, order
accepts an array of two-element arrays specifying
the ordering. Field names and directions may be strings or symbols.
Another way of providing the order is to use #asc
and #desc
methods
on symbols, as follows:
The arguments can be provided as a string using SQL syntax:
Finally, there are asc
and desc
methods that can be used instead of
order
/order_by
:
order
calls can be chained, in which case the oldest calls define the
most significant criteria and the newest calls define the least significant
ones (since in Ruby hashes maintain the order of their keys):
This can sometimes lead to surprising results if there are scopes, including
the default scope, that use order
/order_by
. For example, in the
following snippet bands are ordered by name first because the order in the
default scope takes precedence over the order given in the query, due to
the default scope being evaluated first:
Pagination¶
Mongoid provides the pagination operators limit
, skip
, and batch_size
on Criteria
.
limit
¶
limit
sets the total number of documents to be returned by a query:
skip
¶
skip
(alias: offset
) sets the number of query results to skip
before returning documents. The limit
value, if specified, will be applied
after documents are skipped. When performing pagination, skip
is recommended
to be combined with ordering to ensure consistent results.
batch_size
¶
When executing large queries, or when iterating over query results with an enumerator method such as
Criteria#each
, Mongoid automatically uses the MongoDB getMore command to load results in batches.
The default batch_size
is 1000, however you may set it explicitly:
Query Cache¶
When the query cache is enabled, Mongoid will cache results for MongoDB queries when the entire result set is returned in a single batch (1000 documents by default).
Each thread has its own query cache.
When the query cache is enabled, performing most write operations (insert, update, replace or delete) clears the cache of the thread issuing the write.
To enable the query cache manually for a code segment, use:
The query cache can also be explicitly enabled and disabled, although it is recommended to use the block form described above:
Mongoid also provides a Rack middleware to enable the query cache automatically for each web request.
The Improved Driver Query Cache¶
The Mongoid query cache has been reimplemented in version 2.14.0 of the MongoDB Ruby Driver. The driver query cache is more correct and more effective, and we recommended that you upgrade to version 2.14.0 of the Ruby driver or newer.
For the purposes of this tutorial, the Mongoid query cache will be called “the legacy query cache”, and the driver query cache will be referred to as “the driver query cache.”
With driver versions 2.14.0 or newer, Mongoid will use the driver query cache instead of the legacy query cache. When used with older versions of the driver (that do not implement the query cache) Mongoid will fall back on the legacy query cache.
Mongoid will retain the interface (described above) for enabling and disabling the query cache. When using driver versions 2.14.0 or newer, this interface will affect the driver query cache.
Read more about the Ruby driver query cache in the driver documentation.
Warning
The legacy Mongoid query cache has been deprecated in favor of the query cache implemented in version 2.14.0 of the MongoDB Ruby driver. While the legacy query cache will continue to function, the driver query cache is more correct and more effective. If you plan on using the query cache, it is recommended that you upgrade to Ruby driver version 2.14.0 or newer.
Read more about the Ruby driver query cache in the driver documentation.
Legacy Query Cache Limitations¶
The following is a list of limitations of the legacy query cache:
- The legacy query cache does not cache query results that exceed the batch size. The default batch size is 1000 documents.
- The legacy query cache does not take into account read preference or read concern when deciding whether to return cached results. When performing the same query with a different read preference or read concern with the query cache enabled, incorrect results may be returned.
- Bulk writes, as well as
$out
and$merge
aggregation pipeline stages do not invalidate the query cache. Cached results may be stale after performing any of these operations. - Aggregation results are not cached.
A Note on using #first
¶
Calling the first
method on a model class imposes an ascending sort by
the _id
field on the underlying query. This may produce unexpected behavior
with query caching.
For example, when calling all
on a model class and then first
,
one would expect the second query to use the cached results from the first.
However, because of the sort imposed on the second query, both methods
will query the database and separately cache their results.
To use the cached results, call all.to_a.first
on the model class.
System Collections and the Query Cache¶
MongoDB stores system information in collections that use the database.system.*
namespace pattern. These are called system collections.
Data in system collections can change due to activity not triggered by the application (such as internal server processes) and as a result of a variety of database commands issued by the application. Because of the difficulty of determining when the cached results for system collections should be expired, queries on system collections bypass the query cache.
Neither the legacy query cache nor the driver query cache will cache query results from system collections.
Finding By _id
¶
Mongoid provides the find
method on Criteria
objects to find documents
by their _id
values:
The find
method performs type conversion, if necessary, of the argument
to the type declared in the model being queried for the _id
field.
By default, the _id
type is BSON::ObjectId
, thus the query above
is equivalent to:
Note
When querying collections directly using the driver, type conversion is not automatically performed:
The find
method can accept multiple arguments, or an array of arguments.
In either case each of the arguments or array elements is taken to be an _id
value, and documents with all of the specified _id
values are returned in
an array:
If the same _id
value is given more than once, the corresponding document
is only returned once:
The documents returned are not ordered, and may be returned in a different
order from the order of provided _id
values, as illustrated in the above
examples.
If any of the _id
values are not found in the database, the behavior of
find
depends on the value of the raise_not_found_error
configuration
option. If the option is set to true
, find
raises
Mongoid::Errors::DocumentNotFound
if any of the _id``s are not found.
If the option is set to ``false
and find
is given a single _id
to
find and there is no matching document, find
returns nil
. If the
option is set to false
and find
is given an array of ``_id``s to find
and some are not found, the return value is an array of documents that were
found (which could be empty if no documents were found at all).
Additional Query Methods¶
Mongoid also has some helpful methods on criteria.
Operation | Example |
---|---|
Get the total number of documents matching a filter, or the total number of documents in a collection. Note this will always hit the database for the count. As of Mongoid 7.2, the |
|
Get an approximate number of documents in the collection using the
collection metadata. The |
|
Get a list of distinct values for a single field. Note this will always hit the database for the distinct values. |
|
Iterate over all matching documents in the criteria. |
|
Determine if any matching documents exist. Will return true if there are 1 or more. |
|
Find a document by the provided attributes. If not found,
raise an error or return nil depending on the value of the
|
|
Find a document by the provided attributes, and if not found create and return a newly persisted one. Note that attributes provided in the arguments to this method will override any set in ``create_with``. |
|
Find a document by the provided attributes, and if not found return a new one. |
|
Finds a single document given the provided criteria. This automatically adds a sort on id. Opt out of adding the id sort with the {id_sort: :none} option. |
|
Find the first document by the provided attributes, and if not found create and return a newly persisted one. |
|
Find the first document by the provided attributes, and if not found
create and return a newly persisted one using |
|
Find the first document by the provided attributes, and if not found return a new one. |
|
Find documents for a provided JavaScript expression, optionally with
the specified variables added to the evaluation scope. The scope
argument is supported in MongoDB 4.2 and lower.
In MongoDB 3.6 and higher, prefer $expr over |
|
Same as count but caches subsequent calls to the database |
|
Get all the values for the provided field. Returns nil for unset fields and for non-existent fields. |
Eager Loading¶
Mongoid provides a facility to eager load documents
from associations to prevent the n+1 issue when
iterating over documents with association access. Eager loading is supported on
all associations with the exception of polymorphic belongs_to
associations.
Regular Expressions¶
MongoDB, and Mongoid, allow querying documents by regular expressions.
Given the following model definitions:
… we can query using simple Ruby regular expressions in a natural way:
It is also possible to query using PCRE syntax by constructing
BSON::Regexp::Raw
objects explicitly:
Conditions On Fields¶
When a condition uses a field defined in the model, the value being specified
in the condition is converted according to the rules of the field, if any.
For example, consider the following model definition that contains a Time
field, a Date
field and an implicit Object
field, and also
intentionally does not define a field called deregistered_at
:
Queries on born_on
and registered_at
fields using Date
and Time
values, respectively, are straightforward:
But, note the differences in behavior when providing a Date
instance
in all possible scenarios:
When using the registered_at
field which is of type Time
, the date
was interpreted to be in local time (as per the configured time zone). When using the born_on
field which is of type Date
,
the date was interpreted to be in UTC. When using the voted_at
field
which was defined without a type (hence implicitly as an Object
),
the date was used unmodified in the constructed query. When using a
nonexistent field deregistered_at
the date was interpreted to be in UTC
and converted to a time, matching the behavior of querying a Date
field.
Reloading¶
Use the reload
method to fetch the most recent version of a document from
the database:
If the model has a shard key defined, the shard key value is included in the reloading query.
Note
reload
also works when the document has not been persisted, in which case
it performs a query using the id
value (and shard key value, if a shard
key is defined):
Queries + Persistence¶
Mongoid supports persistence operations off of criteria in a light capacity for when you want to expressively perform multi document inserts, updates, and deletion.
Warning
Criteria ordering and pagination conditions, including order
, limit
,
offset
, and batch_size
, will be ignored on the following operations.
Operation | Example |
---|---|
Create a newly persisted document. |
|
Create a newly persisted document and raise an exception on validation failure. |
|
Create a new (unsaved) document. |
|
Update attributes of the first matching document. |
|
Update attributes of all matching documents. |
|
Perform an $addToSet on all matching documents. |
|
Perform a $bit on all matching documents. |
|
Perform an $inc on all matching documents. |
|
Perform a $pop on all matching documents. |
|
Perform a $pull on all matching documents. |
|
Perform a $pullAll on all matching documents. |
|
Perform a $push on all matching documents. |
|
Perform a $push with $each on all matching documents. |
|
Perform a $rename on all matching documents. |
|
Perform a $set on all matching documents. |
|
Perform a $unset on all matching documents. |
|
Deletes all matching documents in the database. |
|
Deletes all matching documents in the database while running callbacks for all. This loads all documents into memory and can be an expensive operation. |
Scoping¶
Scopes provide a convenient way to reuse common criteria with more business domain style syntax.
Named Scopes¶
Named scopes are simply criteria defined at class load that are referenced by a provided name. Just like normal criteria, they are lazy and chainable.
Named scopes can take procs and blocks for accepting parameters or extending functionality.
By default, Mongoid allows defining a scope that would shadow an existing class method, as the following example shows:
To have Mongoid raise an error when a scope would overwrite an existing class
method, set the scope_overwrite_exception
configuration option to true
.
Default Scopes¶
Default scopes can be useful when you find yourself applying the same criteria to most queries, and wish to specify these criteria as the default. Default scopes are procs that return criteria objects.
Specifying a default scope also initializes the fields of new models to the values given in the default scope, if the values are simple literals:
Note that if a default value is provided both in the field definition and in the default scope, the value in the default scope takes precedence:
Because a default scope initializes fields in new models as just described, defining a default scope with a dotted key and a simple literal value is not possible:
A workaround is to define the default scope as a complex query:
You can tell Mongoid not to apply the default scope by using
unscoped
, which can be inline or take a block.
You can also tell Mongoid to explicitly apply the default scope again later to always ensure it’s there.
If you are using a default scope on a model that is part of an association, you must reload the association to have scoping reapplied. This is important to note if you change a value of a document in the association that would affect its visibility within the scoped association.
Class Methods¶
Class methods on models that return criteria objects are also treated like scopes, and can be chained as well.
Map/Reduce¶
Mongoid provides a DSL around MongoDB’s map/reduce framework, for performing custom map/reduce jobs or simple aggregations.
Execution¶
You can tell Mongoid off the class or a criteria to perform a map/reduce
by calling map_reduce
and providing map and reduce javascript
functions.
Just like criteria, map/reduce calls are lazily evaluated. So nothing will hit the database until you iterate over the results, or make a call on the wrapper that would need to force a database hit.
The only required thing you provide along with a map/reduce is where to
output the results. If you do not provide this an error will be raised.
Valid options to #out
are:
inline: 1
: Don’t store the output in a collection.replace: "name"
: Store in a collection with the provided name, and overwrite any documents that exist in it.merge: "name"
: Store in a collection with the provided name, and merge the results with the existing documents.reduce: "name"
: Store in a collection with the provided name, and reduce all existing results in that collection.
Raw Results¶
Results of Map/Reduce execution can be retrieved via the execute
method
or its aliases raw
and results
:
Statistics¶
MongoDB servers 4.2 and lower provide Map/Reduce execution statistics. As of MongoDB 4.4, Map/Reduce is implemented via the aggregation pipeline and statistics described in this section are not available.
The following methods are provided on the MapReduce
object:
counts
: Number of documents read, emitted, reduced and output through the pipeline.input
,emitted
,reduced
,output
: individual count methods. Note thatemitted
andreduced
methods are named differently from hash keys incounts
.time
: The time, in milliseconds, that Map/Reduce pipeline took to execute.
The following code illustrates retrieving the statistics:
Note
Each statistics method invocation re-executes the Map/Reduce pipeline.
The results of execution are not stored by Mongoid. Consider using the
execute
method to retrieve the raw results and obtaining the statistics
from the raw results if multiple statistics are desired.
Text Search¶
MongoDB provides text indexes to support text search queries on string content. Text indexes can include any field whose value is a string or an array of string elements.
Note
MongoDB Atlas also provides Atlas Search which is a more powerful and flexible text search solution. The rest of this section discusses text indexes and not Atlas Search.
To perform text search with Mongoid, follow these steps:
- Define a text index on a model.
- Create the text index on the server.
- Build a text search query.
Defining Text Search Index¶
Index definition through Mongoid is described in detail on the indexes page. Text search indexes are described in detail under text indexes in the MongoDB manual. Below is an example definition of a Band model with a text index utilizing the description field:
Note that the index type (text
) must be given as a string, not as a symbol.
Creating Text Index¶
To create the index, invoke the db:mongoid:create_indexes
Rake task:
Querying Using Text Index¶
To find bands whose description contains “ounces” or its variations, use the $text operator:
Note that the description contains the word “ounce” even though the search query was “ounces”.
Note also that when performing text search, the name of the field is not
explicitly specified - $text
operator searches all fields indexed with
the text index.