- Reference >
- Database Commands >
- Aggregation Commands >
- mapReduce
mapReduce¶
On this page
-
mapReduce
¶ The
mapReduce
command allows you to run map-reduce aggregation operations over a collection. ThemapReduce
command has the following prototype form:Pass the name of the collection to the
mapReduce
command (i.e.<collection>
) to use as the source documents to perform the map reduce operation. The command also accepts the following parameters:Field Type Description mapReduce
collection The name of the collection on which you want to perform map-reduce. map
Javascript function A JavaScript function that associates or “maps” a
value
with akey
and emits thekey
and valuepair
.See Requirements for the map Function for more information.
reduce
JavaScript function A JavaScript function that “reduces” to a single object all the
values
associated with a particularkey
.See Requirements for the reduce Function for more information.
out
string or document Specifies the location of the result of the map-reduce operation. You can output to a collection, output to a collection with an action, or output inline. You may output to a collection when performing map reduce operations on the primary members of the set; on secondary members you may only use the
inline
output.See out Options for more information.
query
document Optional. Specifies the selection criteria using query operators for determining the documents input to the map
function.sort
document Optional. Sorts the input documents. This option is useful for optimization. For example, specify the sort key to be the same as the emit key so that there are fewer reduce operations. The sort key must be in an existing index for this collection. limit
number Optional. Specifies a maximum number of documents to return from the collection. finalize
Javascript function Optional. Follows the
reduce
method and modifies the output.See Requirements for the finalize Function for more information.
scope
document Optional. Specifies global variables that are accessible in the map
,reduce
andfinalize
functions.jsMode
Boolean Optional. Specifies whether to convert intermediate data into BSON format between the execution of the
map
andreduce
functions. Defaults to false.If
false
:- Internally, MongoDB converts the JavaScript objects emitted
by the
map
function to BSON objects. These BSON objects are then converted back to JavaScript objects when calling thereduce
function. - The map-reduce operation places the intermediate BSON objects in temporary, on-disk storage. This allows the map-reduce operation to execute over arbitrarily large data sets.
If
true
:- Internally, the JavaScript objects emitted during
map
function remain as JavaScript objects. There is no need to convert the objects for thereduce
function, which can result in faster execution. - You can only use
jsMode
for result sets with fewer than 500,000 distinctkey
arguments to the mapper’semit()
function.
The
jsMode
defaults to false.verbose
Boolean Optional. Specifies whether to include the timing
information in the result information. Theverbose
defaults totrue
to include thetiming
information.The following is a prototype usage of the
mapReduce
command:JavaScript in MongoDB
Although
mapReduce
uses JavaScript, most interactions with MongoDB do not use JavaScript but use an idiomatic driver in the language of the interacting application.- Internally, MongoDB converts the JavaScript objects emitted
by the
Important
When connected to a mongos
for a sharded
cluster, to use the mapReduce
directly, you must
specify the all-lower-case form of the command (i.e.``mapreduce``.)
Note
Changed in version 2.4.
In MongoDB 2.4, map-reduce operations
, the
group
command, and $where
operator expressions
cannot access certain global functions or properties, such as
db
, that are available in the mongo
shell.
When upgrading to MongoDB 2.4, you will need to refactor your code if
your map-reduce operations
, group
commands, or $where
operator expressions include any global
shell functions or properties that are no longer available, such as
db
.
The following JavaScript functions and properties are available to
map-reduce operations
, the group
command, and $where
operator expressions in MongoDB 2.4:
Available Properties | Available Functions | |
---|---|---|
args MaxKey MinKey |
assert() BinData() DBPointer() DBRef() doassert() emit() gc() HexData() hex_md5() isNumber() isObject() ISODate() isString() |
Map() MD5() NumberInt() NumberLong() ObjectId() print() printjson() printjsononeline() sleep() Timestamp() tojson() tojsononeline() tojsonObject() UUID() version() |
Requirements for the map
Function¶
The map
function has the following prototype:
The map
function exhibits the following behaviors:
In the
map
function, reference the current document asthis
within the function.The
map
function should not access the database for any reason.The
map
function should be pure, or have no impact outside of the function (i.e. side effects.)The
emit(key,value)
function associates thekey
with avalue
.A single emit can only hold half of MongoDB’s maximum BSON document size.
The
map
function can callemit(key,value)
any number of times, including 0, per each input document.The following
map
function may callemit(key,value)
either 0 or 1 times depending on the value of the input document’sstatus
field:The following
map
function may callemit(key,value)
multiple times depending on the number of elements in the input document’sitems
field:
The
map
function can access the variables defined in thescope
parameter.
Requirements for the reduce
Function¶
The reduce
function has the following prototype:
The reduce
function exhibits the following behaviors:
- The
reduce
function should not access the database, even to perform read operations. - The
reduce
function should not affect the outside system. - MongoDB will not call the
reduce
function for a key that has only a single value. Thevalues
argument is an array whose elements are thevalue
objects that are “mapped” to thekey
. - MongoDB can invoke the
reduce
function more than once for the same key. In this case, the previous output from thereduce
function for that key will become one of the input values to the nextreduce
function invocation for that key. - The
reduce
function can access the variables defined in thescope
parameter.
Because it is possible to invoke the reduce
function
more than once for the same key, the following
properties need to be true:
the type of the return object must be identical to the type of the
value
emitted by themap
function to ensure that the following operations is true:the
reduce
function must be idempotent. Ensure that the following statement is true:the order of the elements in the
valuesArray
should not affect the output of thereduce
function, so that the following statement is true:
out
Options¶
You can specify the following options for the out
parameter:
Output to a Collection¶
Output to a Collection with an Action¶
This option is only available when passing out
a collection that
already exists. This option is not available on secondary members of
replica sets.
When you output to a collection with an action, the out
has the
following parameters:
<action>
: Specify one of the following actions:replace
Replace the contents of the
<collectionName>
if the collection with the<collectionName>
exists.merge
Merge the new result with the existing result if the output collection already exists. If an existing document has the same key as the new result, overwrite that existing document.
reduce
Merge the new result with the existing result if the output collection already exists. If an existing document has the same key as the new result, apply the
reduce
function to both the new and the existing documents and overwrite the existing document with the result.
db
:Optional.The name of the database that you want the map-reduce operation to write its output. By default this will be the same database as the input collection.
sharded
:Optional. If
true
and you have enabled sharding on output database, the map-reduce operation will shard the output collection using the_id
field as the shard key.nonAtomic
:New in version 2.2.
Optional. Specify output operation as non-atomic and is valid only for
merge
andreduce
output modes which may take minutes to execute.If
nonAtomic
istrue
, the post-processing step will prevent MongoDB from locking the database; however, other clients will be able to read intermediate states of the output collection. Otherwise the map reduce operation must lock the database during post-processing.
Output Inline¶
Perform the map-reduce operation in memory and return the result. This
option is the only available option for out
on secondary members of
replica sets.
The result must fit within the maximum size of a BSON document.
Requirements for the finalize
Function¶
The finalize
function has the following prototype:
The finalize
function receives as its arguments a key
value and the reducedValue
from the reduce
function. Be
aware that:
- The
finalize
function should not access the database for any reason. - The
finalize
function should be pure, or have no impact outside of the function (i.e. side effects.) - The
finalize
function can access the variables defined in thescope
parameter.
Examples¶
In the mongo
shell, the db.collection.mapReduce()
method is a wrapper around the mapReduce
command. The
following examples use the db.collection.mapReduce()
method:
Consider the following map-reduce operations on a collection
orders
that contains documents of the following prototype:
Return the Total Price Per Customer¶
Perform the map-reduce operation on the orders
collection to group
by the cust_id
, and calculate the sum of the price
for each
cust_id
:
Define the map function to process each input document:
- In the function,
this
refers to the document that the map-reduce operation is processing. - The function maps the
price
to thecust_id
for each document and emits thecust_id
andprice
pair.
- In the function,
Define the corresponding reduce function with two arguments
keyCustId
andvaluesPrices
:- The
valuesPrices
is an array whose elements are theprice
values emitted by the map function and grouped bykeyCustId
. - The function reduces the
valuesPrice
array to the sum of its elements.
- The
Perform the map-reduce on all documents in the
orders
collection using themapFunction1
map function and thereduceFunction1
reduce function.This operation outputs the results to a collection named
map_reduce_example
. If themap_reduce_example
collection already exists, the operation will replace the contents with the results of this map-reduce operation:
Calculate Order and Total Quantity with Average Quantity Per Item¶
In this example, you will perform a map-reduce operation on the
orders
collection for all documents that have an ord_date
value greater than 01/01/2012
. The operation groups by the
item.sku
field, and calculates the number of
orders and the total quantity ordered for each sku
. The operation concludes by
calculating the average quantity per order for each sku
value:
Define the map function to process each input document:
- In the function,
this
refers to the document that the map-reduce operation is processing. - For each item, the function associates the
sku
with a new objectvalue
that contains thecount
of1
and the itemqty
for the order and emits thesku
andvalue
pair.
- In the function,
Define the corresponding reduce function with two arguments
keySKU
andcountObjVals
:countObjVals
is an array whose elements are the objects mapped to the groupedkeySKU
values passed by map function to the reducer function.- The function reduces the
countObjVals
array to a single objectreducedValue
that contains thecount
and theqty
fields. - In
reducedVal
, thecount
field contains the sum of thecount
fields from the individual array elements, and theqty
field contains the sum of theqty
fields from the individual array elements.
Define a finalize function with two arguments
key
andreducedVal
. The function modifies thereducedVal
object to add a computed field namedavg
and returns the modified object:Perform the map-reduce operation on the
orders
collection using themapFunction2
,reduceFunction2
, andfinalizeFunction2
functions.This operation uses the
query
field to select only those documents withord_date
greater thannew Date(01/01/2012)
. Then it output the results to a collectionmap_reduce_example
. If themap_reduce_example
collection already exists, the operation will merge the existing contents with the results of this map-reduce operation.
For more information and examples, see the Map-Reduce page and Perform Incremental Map-Reduce.