Release Notes¶

• Adds support for Atlas Clusters as a data source.
• Improves performance for the $lookup aggregation pipeline stage.
• Adds support for evaluating string $convert expressions in the filename for $out to S3 .
• Updates Parquet support for MAP types.
• Improves error messaging for $out to S3 .
• Adds a command to generate a storage configuration.

• Automates storage configuration generation for newly created Atlas Data Lakes.
• Allows write partitioning-aware data to S3 using the pipe.$out in Atlas Data Lake.

• Generates Storage Configs when Atlas creates a Atlas Data Lake.
• Adds support for pipe.$out to S3 .
• Updates support for Apache Parquet LIST element.
• Upgrades wire protocol support to 4.2 from 3.6.
• Adds support for verbosity in the explain plan.

• Fixes stability issues.

• Improves performance.
• Supports the $currentOp stage so that you can monitor query progress on long-running queries.
• Updates the isodate attribute to accept additional formats.
• Refreshes the metadata catalog when you use Storage Configuration commands.

• Includes various performance and stability improvements.
• Supports filename field references for pipe.$out.
• Supports $toString in pipe.$out to S3 .

• Supports optionally granting Atlas Data Lake write access to S3 buckets, enabling use of pipe.$out semantics to write directly to those buckets.
• Adds incremental store, database, collection, and view commands for storage configuration management.
• Limits collections returned for wildcard collections to 1,000.
• Updates the storage configuration format.

• Supports cross-database pipe.$lookup queries.
• Supports lowercase and uppercase file extensions.
• Template segments now support dot-separated attribute names that correspond to nested fields.

• Allows the defaultFormat to be specified without a leading dot.
• Supports filtering based on stripes for files in ORC format.
• Allows query attributes to be extracted after the first stage.

• Includes several performance and stability improvements.

• Supports partition definition for the following:

  • epoch_secs, which is seconds since the Unix Epoch
  • epoch_millis, which is milliseconds since the Unix Epoch
  • UUID, which is binary subtype 4

• Includes several performance and stability improvements.
• Adds support for reading Apache ORC files.

• Supports filtering partitions by Parquet file row group statistics.
• Supports ObjectIds in the path when specifying partition databases.<database>.<collection>.[n].definition.

• Returns an error if a query produces a document larger than 16 MiB.
• The pipe.$indexStats stage now produces an empty list of indexes instead of an error.
• Supports pipe.$out to S3 storage format in JSON.
• pipe.$match now implicitly treats all terms as conjunctions.
• No longer parses empty files.
• Fixes an issue that caused the {$match: {$expr: {$and: []}}} expression to terminate the connection.

• Allows nested fields in partition definitions.
• No longer enumerates directories on S3 when a single subdirectory containing all the partitions matching the query is identified.
• Fixes an issue where the new storage configuration did not appear on the issuing connection after running setStorageConfig.

• Adds support for the dbcmd.getLastError database command.
• Fixes a bug with how union types are handled in Avro.
• Supports pipe.$out aggregation pipeline stage to S3.
• dbcmd.listIndexes now always returns an empty list.
• Translates dot-delimited CSV and TSV keys into subdocuments.
• Storage configuration error message now includes a link to the documentation.
• Supports the XLSX file format.
• Includes the correlation ID in query execution error messages.
• Returns an error to the client when the cursor storage limit is reached.
• Returns an error to the client on the last dbcmd.getMore if the cursor storage limit is exceeded.

• Supports dbcmd.listCommands. For example: db.runCommand({"listCommands": 1})
• Includes partition size information in the output of db.collection.explain.

• Returns the first batch of cursor results more quickly.
• Improves performance of pipe.$lookup when combined with pipe.$unwind.
• Automatically supports SCRAM-SHA-1 credentials without requiring drivers to specify this authentication mechanism.
• Provides a descriptive error message when the file format is unknown.
• Provides additional validation on setStorageConfig.

Initial public beta release of Atlas Data Lake.