Docs Menu

Partition Strategies

On this page

  • Overview
  • Firehose Strategy
  • User Strategy
  • Team Strategy
  • Channel Strategy
  • Region Strategy
  • Bucket Strategy

A partition strategy is an approach to configuring how your app partitions data in order to handle a particular use case. This page describes common partition strategies that you can adapt and combine to fit your application.

Realm uses partitions to map objects between synced realms on client devices and documents stored in MongoDB Atlas. Partitions are a core part of your synced data model that determine each user's read and write access to objects in the partition.

When developing a partition strategy, there are two primary areas to consider:

  • Data Security: Your users likely need individualized read and write access to data depending on some conditions. A partition strategy should consider which users need to access which partitions to make sure that a given user always has the same data access permissions for every document in a partition.
  • Storage Capacity: Your client applications may have limited on-device storage space available to them depending on the device and platform. A partition strategy should take storage limitations into account to make sure a user's synced data can be stored on their device.
Tip
Combine Strategies

To use multiple strategies in the same app, you can use a generic partition key field name like _partition with a partition value structured like a query string. For example:

_partitionKey: "user_id=abcdefg"
_partitionKey: "team_id=1234&city='New York, NY'"

You can use functions and rule expressions to parse the string and determine whether or not a given user has access to a realm based on the combined strategy.

A firehose partition strategy groups all documents into a single partition so that every user syncs all of your app's data to their device. This approach is functionally a decision not to partition data and is suitable only for basic applications or relatively small public data sets.

  • Data Security: All data in the firehose partition is public. If a given user has read or write access to the partition, they can read or write any document.
  • Storage Capacity: Every user syncs every document in the firehose partition, so all data must fit within your most restrictive storage limitation. You should only use this strategy for data sets that are relatively small and that do not grow quickly.
Tip

The most straightforward way to configure a firehose is to set the partition key as an optional field not present in any document. Realm maps any document that doesn't include a partition value to the null partition.

Example

A local sports app lets users download and browse game scores and statistics for historical high school baseball games. Consider the following documents in the games and teams collections:

collection games: [
{ teams: ["Brook Ridge Miners", "Southside Rockets"], score: { ... }, date: ... }
{ teams: ["Brook Ridge Miners", "Uptown Bombers"], score: { ... }, date: ... }
{ teams: ["Brook Ridge Miners", "Southside Rockets"], score: { ... }, date: ... }
{ teams: ["Southside Rockets", "Uptown Bombers"], score: { ... }, date: ... }
{ teams: ["Brook Ridge Miners", "Uptown Bombers"], score: { ... }, date: ... }
{ teams: ["Southside Rockets", "Uptown Bombers"], score: { ... }, date: ... }
]
collection teams: [
{ name: "Brook Ridge Miners" }
{ name: "Southside Rockets" }
{ name: "Uptown Bombers" }
]

The total number of historical games is relatively small, and a small number of local teams only play a few games each year, so most devices should be able to download all of the game data for easy offline access. In this case, the firehose strategy is appropriate, and the documents don't need to have a partition key.

The strategy maps the collections to the following realms:

realm null: [
Game { teams: ["Brook Ridge Miners", "Southside Rockets"], score: { ... }, date: ... }
Game { teams: ["Brook Ridge Miners", "Uptown Bombers"], score: { ... }, date: ... }
Game { teams: ["Brook Ridge Miners", "Southside Rockets"], score: { ... }, date: ... }
Game { teams: ["Southside Rockets", "Uptown Bombers"], score: { ... }, date: ... }
Game { teams: ["Brook Ridge Miners", "Uptown Bombers"], score: { ... }, date: ... }
Game { teams: ["Southside Rockets", "Uptown Bombers"], score: { ... }, date: ... }
Team { name: "Brook Ridge Miners" }
Team { name: "Southside Rockets" }
Team { name: "Uptown Bombers" }
]

A user partition strategy groups private documents that belong to a user into a partition specific to that user. This approach is effective when each document has a specific owner, often identified by a username or ID, and no other users need access to the data.

  • Data Security: Data in a user partition is specific to a given user and may contain information private to that user. In general, each user syncs only their own partition and other users cannot access documents in the partition.
  • Storage Capacity: In general, each user only syncs data from their own partition, so their data must fit within their device's storage constraints. You can use this strategy when data has specific owners that each own a manageable amount of data.
Example

A music streaming app stores data about playlists and song ratings for each user. Consider the following documents in the playlists and ratings collections:

collection playlists: [
{ name: "Work", owner_id: "dog_enthusiast_95", song_ids: [ ... ] }
{ name: "Party", owner_id: "cat_enthusiast_92", song_ids: [ ... ] }
{ name: "Soup Tunes", owner_id: "dog_enthusiast_95", song_ids: [ ... ] }
{ name: "Disco Anthems", owner_id: "PUBLIC", song_ids: [ ... ] }
{ name: "Deep Focus", owner_id: "PUBLIC", song_ids: [ ... ] }
]
collection ratings: [
{ owner_id: "dog_enthusiast_95", song_id: 3, rating: -1 }
{ owner_id: "cat_enthusiast_92", song_id: 1, rating: 1 }
{ owner_id: "dog_enthusiast_95", song_id: 1, rating: 1 }
]

Every document includes the owner_id field, which is a good partition key for a user partition strategy because it naturally maps documents to individual users. This limits the data on each device to only include playlists and ratings that are relevant for the device's user.

  • Users have read and write access to their user realm, which contains playlists they have created and ratings that they've given to songs.
  • Every user has read access to the realm for partition value PUBLIC, which contains playlists that are available to all users.

The strategy maps the collections to the following realms:

realm dog_enthusiast_95: [
Playlist { name: "Work", owner_id: "dog_enthusiast_95", song_ids: [ ... ] }
Playlist { name: "Soup Tunes", owner_id: "dog_enthusiast_95", song_ids: [ ... ] }
Rating { owner_id: "dog_enthusiast_95", song_id: 3, rating: -1 }
Rating { owner_id: "dog_enthusiast_95", song_id: 1, rating: 1 }
]
realm cat_enthusiast_92: [
Playlist { name: "Party", owner_id: "cat_enthusiast_92", song_ids: [ ... ] }
Rating { owner_id: "cat_enthusiast_92", song_id: 1, rating: 1 }
]
realm PUBLIC: [
Playlist { name: "Disco Anthems", owner_id: "PUBLIC", song_ids: [ ... ] }
Playlist { name: "Deep Focus", owner_id: "PUBLIC", song_ids: [ ... ] }
]

A team partition strategy groups private documents shared by a team of users (e.g. a store location's employee, a band's members, etc) into a partition specific to that group. All users in the team share access and ownership of the team's documents.

  • Data Security: Data in a team partition is specific to a given team and may contain information private to that team but not necessarily private to any specific member of the team. In general each user syncs partitions for the teams they belong to and users not in a given team cannot access documents in the team's partition.
  • Storage Capacity: In general, each user only syncs data from their own teams. The combined data from any given user's teams must fit within their device's storage constraints. You can use this strategy when data belongs to specific teams and each user belongs to a manageable number of teams. If a user may belong to many teams, you may need to limit the number of team partitions that they sync at any time.
Example

A productivity app lets users create projects where they can collaborate on tasks with other users. Consider the following documents in the projects and tasks collections:

collection projects: [
{ name: "CLI", owner_id: "cli-team", task_ids: [ ... ] }
{ name: "API", owner_id: "api-team", task_ids: [ ... ] }
]
collection tasks: [
{ status: "complete", owner_id: "api-team", text: "Import dependencies" }
{ status: "inProgress", owner_id: "api-team", text: "Create app MVP" }
{ status: "inProgress", owner_id: "api-team", text: "Investigate off-by-one issue" }
{ status: "todo", owner_id: "api-team", text: "Write tests" }
{ status: "inProgress", owner_id: "cli-team", text: "Create command specifications" }
{ status: "todo", owner_id: "cli-team", text: "Choose a CLI framework" }
]

Every document includes the owner_id field, which is a good partition key for a team partition strategy because it naturally maps documents to individual teams. This limits the data on each device to only include projects and tasks that are relevant for the device's user.

  • Users have read and write access to partitions owned by a team that they're a member of.
  • Team membership could be determined by data stored in a teams or users collection:

    collection teams: [
    { name: "cli-team", member_ids: [ ... ] }
    { name: "api-team", member_ids: [ ... ] }
    ]
    collection users: [
    { name: "Joe", team_ids: [ ... ] }
    { name: "Liz", team_ids: [ ... ] }
    { name: "Matt", team_ids: [ ... ] }
    { name: "Emmy", team_ids: [ ... ] }
    { name: "Scott", team_ids: [ ... ] }
    ]

The strategy maps the collections to the following realms:

realm cli-team: [
Project { name: "CLI", owner_id: "cli-team", task_ids: [ ... ] }
Task { status: "inProgress", owner_id: "cli-team", text: "Create command specifications" }
Task { status: "todo", owner_id: "cli-team", text: "Choose a CLI framework" }
]
realm api-team: [
Project { name: "API", owner_id: "api-team", task_ids: [ ... ] }
Task { status: "complete", owner_id: "api-team", text: "Import dependencies" }
Task { status: "inProgress", owner_id: "api-team", text: "Create app MVP" }
Task { status: "inProgress", owner_id: "api-team", text: "Investigate off-by-one issue" }
Task { status: "todo", owner_id: "api-team", text: "Write tests" }
]

A channel partition strategy groups documents relevant to a common topic or domain into a partition specific to that area. Users can choose to access or subscribe to specific channels, often identified by a name or ID, from a public list.

  • Data Security: Data in a channel partition is specific to a given topic or area that a user can choose to access. You can limit a user's access to specific channels or prevent them from accessing channels entirely, but if a user has read or write permission for a channel then they can access any document in the partition.
  • Storage Capacity: Users can choose to sync data from any allowed channel. The combined data from all of the channels a given user accesses must fit within their device's storage constraints. You can use this strategy to partition public or semi-private data sets that would otherwise not fit within storage constraints.
Example

A social app lets users create chatrooms based on specific topics. Users can search for and join channels for any topic that interests them. Consider the following documents in the chatrooms and messages collections:

collection chatrooms: [
{ topic: "cats", description: "A place to talk about cats" }
{ topic: "sports", description: "We like sports and we don't care who knows" }
]
collection messages: [
{ topic: "cats", text: "Check out this cute pic of my cat!", timestamp: 1625772933383 }
{ topic: "cats", text: "Can anybody recommend a good cat food brand?", timestamp: 1625772953383 }
{ topic: "sports", text: "Did you catch the game last night?", timestamp: 1625772965383 }
{ topic: "sports", text: "Yeah what a comeback! Incredible!", timestamp: 1625772970383 }
{ topic: "sports", text: "I can't believe how they scored that goal.", timestamp: 1625773000383 }
]

Every document includes the topic field, which is a good partition key for a channel partition strategy because it naturally maps documents to individual channels. This limits the data on each device to only include messages and metadata for chatrooms that the user has subscribed to.

  • Users could have read and write access to chatrooms that they're subscribed to, which would allow them to modify or delete any message, even those sent by other users. To limit write permissions, you could give users read-only access and handle sending messages with a serverless function.
  • You could determine which channels a user has subscribed to by storing the subscription in either the chatrooms or users collection:

    collection chatrooms: [
    { topic: "cats", subscriber_ids: [ ... ] }
    { topic: "sports", subscriber_ids: [ ... ] }
    ]
    collection users: [
    { name: "Joe", chatroom_ids: [ ... ] }
    { name: "Liz", chatroom_ids: [ ... ] }
    { name: "Matt", chatroom_ids: [ ... ] }
    { name: "Emmy", chatroom_ids: [ ... ] }
    { name: "Scott", chatroom_ids: [ ... ] }
    ]

The strategy maps the collections to the following realms:

realm cats: [
Chatroom { topic: "cats", description: "A place to talk about cats" }
Message { topic: "cats", text: "Check out this cute pic of my cat!", timestamp: 1625772933383 }
Message { topic: "cats", text: "Can anybody recommend a good cat food brand?", timestamp: 1625772953383 }
]
realm sports: [
Chatroom { topic: "sports", description: "We like sports and we don't care who knows" }
Message { topic: "sports", text: "Did you catch the game last night?", timestamp: 1625772965383 }
Message { topic: "sports", text: "Yeah what a comeback! Incredible!", timestamp: 1625772970383 }
Message { topic: "sports", text: "I can't believe how they scored that goal.", timestamp: 1625773000383 }
]

A region partition strategy groups documents that are relevant to a specific location or region into partitions specific to those areas.

  • Data Security: Data in a region partition is specific to a given geographic area. You can limit a user's access to just the region they're currently in or give access to data on a region-by-region basis.
  • Storage Capacity: The storage implications of a region strategy depend on the size and usage patterns for each region. Users typically only sync data in their own region, so at a minimum the data for any given region should fit within a device's storage constraints. If users sync multiple regions, you may consider partitioning into smaller subregions to minimize syncing unneeded data.
Example

A food delivery app lets users search for nearby restaurants and order from their menus. Consider the following documents in the restaurants collection:

collection restaurants: [
{ city: "New York, NY", name: "Joe's Pizza", menu: [ ... ] }
{ city: "New York, NY", name: "Han Dynasty", menu: [ ... ] }
{ city: "New York, NY", name: "Harlem Taste", menu: [ ... ] }
{ city: "Chicago, IL", name: "Lou Malnati's", menu: [ ... ] }
{ city: "Chicago, IL", name: "Al's Beef", menu: [ ... ] }
{ city: "Chicago, IL", name: "Nando's", menu: [ ... ] }
]

Every document includes the city field, which is a good partition key for a region partition strategy because it naturally maps documents to specific physical areas. This limits the data on each device to only include messages and metadata for the user's current city. Users have read access to restaurants in their current region, which you could determine in your application logic.

The strategy maps the collections to the following realms:

realm New York, NY: [
{ city: "New York, NY", name: "Joe's Pizza", menu: [ ... ] }
{ city: "New York, NY", name: "Han Dynasty", menu: [ ... ] }
{ city: "New York, NY", name: "Harlem Taste", menu: [ ... ] }
]
realm Chicago, IL: [
{ city: "Chicago, IL", name: "Lou Malnati's", menu: [ ... ] }
{ city: "Chicago, IL", name: "Al's Beef", menu: [ ... ] }
{ city: "Chicago, IL", name: "Nando's", menu: [ ... ] }
]

A bucket partition strategy groups documents that range along a particular dimension into partitions that each contain documents from a discrete subrange. In time-based bucket strategies, you can use scheduled triggers to move documents to a new partition when they fall out of their bucket range.

  • Data Security: You can use permissions to limit users to read or write only specific buckets. Data may flow between buckets, so consider the access permissions for a document across all possible buckets, not just its current bucket.
  • Storage Capacity: The storage implications of a bucket strategy depend on the size and usage patterns for each bucket. Consider which buckets users need to access and limit the size of buckets to fit within a device's storage constraints. If users sync many buckets, especially for disjoint data, consider partitioning into smaller buckets to minimize syncing unneeded data.
Example

An IoT app shows real-time views of sensor readings that come in several times a second. It groups readings into time buckets based on the number of seconds since the reading was taken. Consider the following documents in the readings collection:

collection readings: [
{ bucket: "0s<t<=60s", timestamp: 1625773000383 , data: { ... } }
{ bucket: "0s<t<=60s", timestamp: 1625772970383 , data: { ... } }
{ bucket: "0s<t<=60s", timestamp: 1625772965383 , data: { ... } }
{ bucket: "60s<t<=300s", timestamp: 1625772953383 , data: { ... } }
{ bucket: "60s<t<=300s", timestamp: 1625772933383 , data: { ... } }
]

Every document includes the bucket field, which maps documents to specific time ranges. This limits the data on each device to only include sensor readings for the time period a user is viewing.

  • Users have read access to sensor readings for any time bucket they view
  • The sensors use their own client apps with write access to upload readings

The strategy maps the collections to the following realms:

realm 0s<t<=60s: [
Reading { bucket: "0s<t<=60s", timestamp: 1625773000383 , data: { ... } }
Reading { bucket: "0s<t<=60s", timestamp: 1625772970383 , data: { ... } }
Reading { bucket: "0s<t<=60s", timestamp: 1625772965383 , data: { ... } }
]
realm 60s<t<=300s: [
Reading { bucket: "60s<t<=300s", timestamp: 1625772953383 , data: { ... } }
Reading { bucket: "60s<t<=300s", timestamp: 1625772933383 , data: { ... } }
]
Give Feedback
© 2021 MongoDB, Inc.

About

  • Careers
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2021 MongoDB, Inc.