Navigation

Tailable Cursor Iteration

Overview

When the driver executes a query or command (e.g. aggregate), results from the operation are returned via a MongoDB\Driver\Cursor object. The Cursor class implements PHP’s Iterator interface, which allows it to be iterated with foreach and interface with any PHP functions that work with iterables. Similar to result objects in other database drivers, cursors in MongoDB only support forward iteration, which means they cannot be rewound or used with foreach multiple times.

Tailable cursors are a special type of MongoDB cursor that allows the client to read some results and then wait until more documents become available. These cursors are primarily used with Capped Collections and Change Streams.

While normal cursors can be iterated once with foreach, that approach will not work with tailable cursors. When foreach is used with a tailable cursor, the loop will stop upon reaching the end of the initial result set. Attempting to continue iteration on the cursor with a second foreach would throw an exception, since PHP attempts to rewind the cursor. Therefore, reading from a tailable cursor will require direct usage of the Iterator API.

Note

Before version 1.9.0 of the ext-mongodb extension, the cursor class does not implement the Iterator interface. To manually iterate a cursor using the method below, it must first be wrapped with an IteratorIterator.

Manually Iterating a Normal Cursor

Before looking at how a tailable cursor can be iterated, we’ll start by examining how the Iterator methods interact with a normal cursor.

The following example finds five restaurants and uses foreach to view the results:

<?php

$collection = (new MongoDB\Client)->test->restaurants;

$cursor = $collection->find([], ['limit' => 5]);

foreach ($cursor as $document) {
   var_dump($document);
}

While this example is quite concise, there is actually quite a bit going on. The foreach construct begins by rewinding the iterable ($cursor in this case). It then checks if the current position is valid. If the position is not valid, the loop ends. Otherwise, the current key and value are accessed accordingly and the loop body is executed. Assuming a break has not occurred, the iterator then advances to the next position, control jumps back to the validity check, and the loop continues.

With the inner workings of foreach under our belt, we can now translate the preceding example to use the Iterator methods directly:

<?php

$collection = (new MongoDB\Client)->test->restaurants;

$cursor = $collection->find([], ['limit' => 5]);

$cursor->rewind();

while ($cursor->valid()) {
   $document = $cursor->current();
   var_dump($document);
   $cursor->next();
}

Note

Calling $cursor->next() after the while loop naturally ends would throw an exception, since all results on the cursor have been exhausted.

The purpose of this example is simply to demonstrate the functional equivalence between foreach and manual iteration with PHP’s Iterator API. For normal cursors, there is little reason to manually iterate results instead of a concise foreach loop.

Iterating a Tailable Cursor

In order to demonstrate a tailable cursor in action, we’ll need two scripts: a “producer” and a “consumer”. The producer script will create a new capped collection using MongoDB\Database::createCollection() and proceed to insert a new document into that collection each second.

<?php

$database = (new MongoDB\Client)->test;

$database->createCollection('capped', [
    'capped' => true,
    'size' => 16777216,
]);

$collection = $database->selectCollection('capped');

while (true) {
    $collection->insertOne(['createdAt' => new MongoDB\BSON\UTCDateTime()]);
    sleep(1);
}

With the producer script still running, we will now execute a consumer script to read the inserted documents using a tailable cursor, indicated by the cursorType option to MongoDB\Collection::find(). We’ll start by using foreach to illustrate its shortcomings:

<?php

$collection = (new MongoDB\Client)->test->capped;

$cursor = $collection->find([], [
    'cursorType' => MongoDB\Operation\Find::TAILABLE_AWAIT,
    'maxAwaitTimeMS' => 100,
]);

foreach ($cursor as $document) {
    printf("Consumed document created at: %s\n", $document->createdAt);
}

If you execute this consumer script, you’ll notice that it quickly exhausts all results in the capped collection and then terminates. We cannot add a second foreach, as that would throw an exception when attempting to rewind the cursor. This is a ripe use case for directly controlling the iteration process using the Iterator interface.

<?php

$collection = (new MongoDB\Client)->test->capped;

$cursor = $collection->find([], [
    'cursorType' => MongoDB\Operation\Find::TAILABLE_AWAIT,
    'maxAwaitTimeMS' => 100,
]);

$cursor->rewind();

while (true) {
   if ($cursor->valid()) {
      $document = $cursor->current();
      printf("Consumed document created at: %s\n", $document->createdAt);
   }

   $cursor->next();
}

Much like the foreach example, this version on the consumer script will start by quickly printing all results in the capped collection; however, it will not terminate upon reaching the end of the initial result set. Since we’re working with a tailable cursor, calling next() will block and wait for additional results rather than throw an exception. We will also use valid() to check if there is actually data available to read at each step.

Since we’ve elected to use a TAILABLE_AWAIT cursor, the server will delay its response to the driver for a set amount of time. In this example, we’ve requested that the server block for approximately 100 milliseconds by specifying the maxAwaitTimeMS option to MongoDB\Collection::find().

←   Indexes Example Data  →