Filters and SQL¶

On this page

Filters
SQL

Filters¶

Note

When using filters with DataFrames or the R API, the underlying Mongo Connector code constructs an aggregation pipeline to filter the data in MongoDB before sending it to Spark.

Use filter() to read a subset of data from your MongoDB collection.

Consider a collection named fruit that contains the following documents:

copy

{ "_id" : 1, "type" : "apple", "qty" : 5 }
{ "_id" : 2, "type" : "orange", "qty" : 10 }
{ "_id" : 3, "type" : "banana", "qty" : 15 }

First, set up a dataframe to connect with your default MongoDB data source:

copy

df <- read.df("", source = "com.mongodb.spark.sql.DefaultSource")

Note

The empty argument (“”) refers to a file to use as a data source. In this case our data source is a MongoDB collection, so the data source argument is empty.

The following operation filters the data and includes records where the qty field is greater than or equal to 10:

copy

head(filter(df, df$qty >= 10))

The operation prints the following output:

copy

_id qty   type
1   2  10 orange
2   3  15 banana

SQL¶

Before running SQL queries on your dataset, you must register a temporary view for the dataset.

The following example registers a temporary table called temp, then uses SQL to query for records in which the type field contains the letter e:

copy

createOrReplaceTempView(df, "temp")
some_fruit <- sql("SELECT type, qty FROM temp WHERE type LIKE '%e%'")
head(some_fruit)

In the sparkR shell, the operation prints the following output:

copy

type qty
1  apple   5
2 orange  10

← Aggregation FAQ →