Navigation

Release Notes for MongoDB Connector for BI

MongoDB Connector for BI 2.13.1

Released November 26, 2019

  • The max_num_global_tables system variable now defaults to 4000, raised from 1000.

MongoDB Connector for BI 2.13.0

Released November 11, 2019

Bug Fixes

Fixes the following bugs:

  • unix_timestamp returning incorrect results because it did not handle Daylight Savings Time when necessary.
  • DRDL tables incorrectly marshalling to BSON if the aggregation pipeline was empty.

Improvements

Version 2.13.0 includes the following improvements:

  • Updates an error message for FLUSH SAMPLE authorization that was unclear.
  • Updates schema mapping to skip empty field names and field names containing . characters.
  • Enables self-join optimization for sharded collections.
  • Replaces the max_num_columns_per_table system variable with max_num_fields_per_collection.
  • Enhances performance while sampling collections with many fields by reducing the amount of memory used.
  • Improves ambiguous column behavior in subqueries, GROUP BY and ORDER BY clauses.
  • Adds support for Ubuntu 18.04, SLES 15, and RHEL8.
  • Adds non-string support for regexp.
  • Improves performance for schemas that contain arrays.

MongoDB Connector for BI 2.12.0

Released September 16, 2019

Bug Fixes

Fixes the following bugs:

  • mongosqld hangs if an election occurred during sampling.
  • Some queries using $lookup fail.
  • Panics occur when connecting with a DRDL file that excludes a child table’s _id.
  • Versions 2.10+ install to the wrong directory on Windows.
  • Fixes a bug in the div operator’s in-memory evaluation.
  • mongosqld incorrectly kills long-running queries.
  • Some queries on ObjectId fields do not push down.
  • mongodrdl not parsing replica set seedlists passed to the --host flag.
  • timestampadd fails with out-of-range months.
  • Allows aggregate functions as grouping expressions.
  • datediff fails to push down with polymorphic arguments.
  • Displays incorrect foreign key constraints.
  • Incorrect pushdown and type conversions for date and datetime types that are too large or too small.
  • Aliases for tables in information_schema are ignored.
  • Some queries with aggregate functions fail to push down.
  • Pipeline parsing error for unwind paths with numeric field names.
  • mongodrdl ignores the --gssapiHostName and --gssapiServiceName flags.
  • Allows extra arguments to the current_timestamp and trim functions.
  • str_to_date returns a date instead of a datetime for some non-constant format-string arguments.

Improvements

Pushdown

This release includes a number of improvements to pushdown. Existing translations were improved and expanded pushdown coverage to more functions and query shapes.

  • Adds support for pushing down correlated subqueries to MongoDB versions 3.6 and later.
  • Adds pushdown support for ln, ascii, user, database, version, connection_id, and constant-valued time/date functions.
  • Adds support for pushing down regexp to the aggregation language.
  • Removes some unnecessary rounding from pushdown translations.
  • Adds support for pushing down LIKE expressions with literal pattern strings to the aggregation language.
  • Improves the efficiency of pipelines generated for EXISTS subqueries.
  • Improves truncate pushdown coverage and updated the translation to use $trunc with MongoDB versions 4.1.9 and later.
  • Adds pushdown support for the char and str_to_date functions.

Go Driver

This release includes a number of improvements made possible by upgrading to the new go driver:

  • Adds support for SRV URI support to mongodrdl and mongosqld.
  • Adds MongoDB URI support to mongodrdl via a --uri flag.
  • Adds support for zlib and snappy compression for connections to MongoDB.
  • Updates DRDL file parsing to support the most recent version of the extjson spec.

Behavioral Changes

  • Adds new reserved words to the parser: INSERT, INTO, KEY, FULLTEXT, and PRIMARY.

    Going forward, these words must be wrapped in backticks to be used as identifiers in queries.

  • Removes support for altering sampled schema with ALTER TABLE statements.

  • Prior to 2.12, mongosqld supported renaming and removing tables and columns via a modified ALTER TABLE syntax when in standalone schema mode. Any customer currently using this feature is advised to migrate their schema management needs to views, DRDL, or custom schema mode. The variable/config parameter that enabled this feature, enable_table_alterations, is still supported but doesn’t do anything in BI Connector 2.12. It will be removed in a future release.

MongoDB Connector for BI 2.11.1

Released July 8, 2019

Bug Fixes

Fixes a bug that caused the MySQL schema to include incorrect foreign key constraints.

Fixes a bug that caused incorrect pushdown for some very large or very small date and datetime values.

Fixes a bug that caused aliases to be ignored for tables in the information_schema database.

MongoDB Connector for BI 2.11.0

Released May 16, 2019

New Features

  • Adds a number of optimizations that are applied to generated aggregation pipelines before sending them to MongoDB.
  • Adds new subcommands to mongodrdl for storing a DRDL file in MongoDB and for manipulating stored schemas.
  • Enables mongosqld to use stored DRDL schemas instead of sampling. To learn more, see Schema Management Changes in 2.11.
  • Introduces the beta version of mongotranslate, a command-line utility that can translate SQL queries to aggregation pipelines without a running BI Connector or MongoDB instance.

Bug Fixes

  • Fixes a problem that caused the BI Connector to generate unnecessarily long $group key names.
  • Fixes a bug that caused the BI Connector to generate incorrect table pipelines in polymorphic schema mapping mode.
  • Fixes a bug that caused the BI Connector to panic when printing some query plans.
  • Fixes a bug where the BI Connector’s in-memory evaluation of string-to-date conversions did not match the pushdown behavior.
  • Updates the supported Amazon Linux version to match the version supported by MongoDB 4.2.
  • Fixes a bug with Active Directory constrained delegation in the BI Connector’s GSSAPI implementation (linux and macOS).
  • Introduces a --gssapiConstrainedDelegation flag that must be enabled to ensure existing Kerberos configurations are not broken.
  • Fixes a bug that caused Kerberos authentication to fail with a linux mongosqld, MongoDB 4.0, and an Active Directory KDC.
  • Fixes a number of small bugs related to correlated subqueries.

Chages/Deprecations

  • Changes some schema-management configuration options.
  • Deprecates schema-modification support via ALTER TABLE, as well as the enable_table_alterations system variable and setParameter.

MongoDB Connector for BI 2.10.0

Released Mar 15, 2019

Bug Fixes

  • Fixes a bug where aggregate functions returned NULL in dual queries with the DUAL keyword omitted.
  • Fixes a bug where some invalid DUAL queries did not return an error.
  • Fixes a bug that prevented pushdown of some joins to derived tables selecting from DUAL.
  • Fixes some correctness bugs in the in-memory evaluation for mysql-mode conversions.
  • Fixes a bug where some trig functions (ascii, acos, and asin) could return incorrect results due to missing type conversions.
  • Fixes a bug where some subquery comparisons could return incorrect results due to missing type conversions.
  • Fixes a bug where we lost precision while converting a Uint64 to a Decimal.
  • Fixes a correctness bug with the INSERT scalar function’s pushdown translation.
  • Fixes a bug where unstable sorting caused non-deterministic renaming of colliding collection/field names during schema mapping.
  • Fixes a bug where some valid subquery comparisons returned errors.
  • Fixes a bug where some valid subquery and tuple comparisons returned errors.
  • Fixes a bug where queries using xor returned errors on MongoDB 3.2 because the generated pipelines required features added in MongoDB 3.4.
  • Fixes a bug that caused kerberos authentication to fail with MongoDB 4.0.

Date and Time Changes

This release resolves a number of inconsistencies in the BI Connector’s date and timestamp semantics. Resolving these inconsistencies resulted in some changes to user-facing behavior:

  • Removes inconsistent support for partial dates.

    • MySQL can represent partial dates that are missing month or day components. It is impossible to represent these incomplete dates with MongoDB dates. Attempting to correctly manipulate and render incomplete dates was causing a number of bugs and inconsistencies with complete dates.
    • Starting with version 2.10, the BI Connector does not support operations on or expressions that return incomplete dates. Going forward, we cannot provide guarantees about the behavior of operations that attempt to construct incomplete dates.
  • Updates the DATE(a) and TIMESTAMP(a) scalar functions to be equivalent to CAST(a AS DATE), and CAST(a AS DATETIME), respectively.

    • The behavior of the DATE and TIMESTAMP scalar functions will depend on the value of the type_conversion_mode variable. This will change the default behavior for those scalar functions.

    • Queries affected by this change can be rewritten to use a more explicit conversion function (such as str_to_date) that does not depend on type_conversion_mode. If an affected query cannot be rewritten, a user may set type_conversion_mode variable to mysql, which instructs the BI Connector to perform all type conversions with MySQL semantics instead of MongoDB semantics.

      Important

      This is discouraged unless absolutely necessary, as it significantly curtails the ability to push down type conversions across the board.

    • For more information on type conversions and the type_conversion_mode system variable, see Type Conversion Modes.

MongoDB Connector for BI 2.9.0

Released Dec 18, 2018

Features and Performance Improvements

  • Includes a sample mongosqld configuration file in the release archive.
  • Allows setting commonly used system variables in the configuration file.
  • Optimizes conversions of years to days.
  • Optimizes join performance.
  • General performance improvements.

Bug Fixes

  • Improves the BI Connector’s resiliency to queries against fields that may contain unsupported BSON types.

MongoDB Connector for BI 2.8.2

Released Nov 16, 2018

  • Fixes correctness bugs discovered in polymorphic schema mapping, query cancellation, and IN queries.
  • Fixes a performance regression in a common query pattern.

MongoDB Connector for BI 2.8.1

Released Nov 09, 2018

  • Fixes a bug in the behavior of the max_execution_time system variable.

MongoDB Connector for BI 2.8.0

Released Nov 06, 2018

Features and Performance Improvements

  • Improvements to the BI Connector’s type system and the way it handles polymorphic data.
  • Users can alter the type of a given column.
  • Users can access “dirty” datasets without query errors.
  • Support for pushdown of the GROUP_CONCAT aggregation function.
  • Non-admin users can set global variables.

Bug Fixes

  • Improvements in parsing strToDateTime to float.
  • Fixes to how the BI Connector honors readPreference specifications.

MongoDB Connector for BI 2.7.0

Released Oct 02, 2018

Features and Performance Improvements

  • Collapse associative functions
  • Support the optimize_view_sampling global variable
  • Support the polymorphic_type_conversion_mode session/global variable
  • Improve filter optimization for MongoDB versions 3.6 and later
  • Expose type_conversion_mode in configuration parameters
  • Cache catalog columns for quicker lookup

Bug Fixes

  • Fix incorrect semantics for null safe equality operator ( <=> )
  • The || operator returns incorrect type (and value) when operating on two strings
  • Add DRDL type validation
  • Support CAST to unsigned
  • Fix WHERE clause removed in certain cross-join queries
  • Fix some incorrect left self-join results when dynamic schemas are used
  • Support turning off view optimization with $match
  • ODBC driver on macOS should install in versioned directories
  • Fix ODBC driver installer issue on macOS that caused unicode driver to be reported as ANSI

Released version 1.2.0 of the MongoDB ODBC Driver.

MongoDB Connector for BI 2.6.1

Released Sep 21, 2018

  • Resolved an issue caused by how scalar functions handle string values.

We encourage Tableau users running BIC 2.6.0 to upgrade to 2.6.1.

MongoDB Connector for BI 2.6.0

Released Sep 04, 2018

EXPLAIN Support

Users can use EXPLAIN to return details of aggregation pipeline used to generate query results.

Common Table Expression Support

The BI Connector now supports queries that include Common Table Expressions.

MongoDB Connector for BI 2.5.0

Released May 23, 2018

  • Improved memory management for mongosqld.
  • New SSPI support for mongosqld on Windows.
  • The --sampleNamespaces option now supports exclusion.
  • NULL valued types are now cast to varchar during sampling.
  • Optimized COUNT(*) query performance.
  • SIN, COS, TAN are now pushed down.

New SSPI Support

Windows Clients may now use kerberos to authenticate against the BI Connector.

NULL Valued Types Are Now Cast to Varchar During Sampling

This resolves the issue where queries would fail due to NULL values being ignored during sampling. Now, NULL values will be cast as SQL type VARCHAR.

SIN, COS, TAN Are Now Pushed Down

BI Connector will now push down SIN, COS, and TAN SQL functions. This will improve performance for queries using these functions.

MongoDB Connector for BI 2.4.1

Released Mar 09, 2018

We’ve released version 2.4.1 of the MongoDB Connector for BI for production deployment. This release contains minor bug fixes for users of the BI Connector and is a recommended upgrade for all 2.4 users.

MongoDB Connector for BI 2.4.0

Released Feb 21, 2018

MongoDB Connector for BI version 2.4 improves more expressive queries. New features include:

Aggregate Expressions

BI Connector 2.4 improves support for aggregate expressions including math, date, and string functions.

Improved Support for Subqueries and Non-equijoins

With version 2.4, mongosqld has improved support for subqueries and non-equijoins.

MongoDB Connector for BI 2.3.1

Released Nov 10, 2017

  • Fixes minor incompatibility with MongoDB 3.2.

MongoDB Connector for BI 2.3.0

Released Nov 08, 2017

  • Simplified and centralized Schema Management.
  • Supports log rotation.
  • Standardized logging in the BI Connector.
  • Push LTRIM, RTRIM and TRIM functions down to MongoDB.

MongoDB Connector for BI version 2.3 improves enterprise deployment support. The new features include:

Improved Schema Management

In BI Connector 2.3, mongosqld includes the schema mapping functionality previously found in mongodrdl.

Log Rotation

With version 2.3, you can rotate the mongosqld logs. To rotate the logs, issue a command or restart mongosqld.

MongoDB Connector for BI 2.2.1

Released Aug 29, 2017

  • Fixes lack of support for Read Preference tags which are used when targeting reads to specific members of a replica set.

MongoDB Connector for BI 2.2.0

Released Jul 31, 2017

  • Supports authenticating directly against MongoDB using the new C and JDBC authentication plugins. These plugins support SCRAM-SHA-1 and PLAIN mechanisms and remove the SSL requirement for authentication. The authentication plugins can be found on GitHub:

  • Supports configuration files for mongosqld.

  • Supports installing and running mongosqld as a Windows system service.

  • Provides a Windows MSI installer for the MongoDB Connector for BI.

  • Provides new command line arguments for default auth mechanism and database:

    • Authentication Options:

      --defaultAuthDatabase

      default user source (default of admin)

      --defaultAuthMechanism arg

      default authentication mechanism (default of SCRAM-SHA-1)

  • Provides a new startup option to specify default VARCHAR length

  • Provides mongosqld status metrics with SHOW STATUS

    • Reports the following metrics reported:
      • Connections
      • Queries
      • BytesSent
      • Threads Connected
  • Can bind mongosqld to multiple IP addresses

  • Provides a new runtime options to limit memory usage of GROUP, SORT, and JOIN stages

Authentication Plugins

MongoDB Connector for BI Version 2.0 introduced passthrough authentication, in which clients authenticate as a user defined in MongoDB, without the need to create a separate user within the connector itself.

MongoDB Connector for BI Version 2.2 further increases ease of use with a set of authentication plugins which can authenticate users through the connector without the need for SSL certificates. These plugins perform user authentication via the SASL framework and support the SCRAM-SHA-1 and PLAIN authentication mechanisms. We currently provide authentication plugins in the following languages:

  • Java - for use with MySQL’s Connector/J driver
  • C - for use with the MySQL ODBC driver and the MySQL Shell

MongoDB Connector for BI 2.1.0

  • Provides substantial performance improvements for some common JOIN scenarios.
  • Pushes $match stages earlier in the aggregation pipeline to improve to index utilization.
  • Pushes down more SQL functions and operators to the aggregation pipeline.
  • Supports additional SQL language features, such as USE and SHOW CREATE TABLE statements, STRAIGHT_JOIN, the unary plus (+) operator, and the INTERVAL keyword in date arithmetic.

Note

mongosqld no longer offers the --fork option.

To run mongosqld in the background, use the nohup command:

nohup mongosqld [options] &

MongoDB Connector for BI 2.0.0

Released Nov 29, 2016

Improved Server-Side Execution

The MongoDB Connector for BI delivers significant performance and scalability improvements by moving as much execution as possible to the underlying MongoDB instance. Pushing execution down to the server reduces network latency and reduces bandwidth consumption.

This approach also reduces memory and CPU consumption in the MongoDB Connector for BI itself.

Pass-through Authentication

The user authentication process is now passed through the MongoDB Connector for BI to the underlying MongoDB instance. Users now authenticate as a user already created within MongoDB.

Simplified Architecture

The MongoDB Connector for BI package is now trimmed down from four separate components to two:

mongodrdl
This tool connects to a MongoDB instance and generates a Document-Relational Definition Language (DRDL) file, which maps a given collection shape to relational schema.
mongosqld

Once installed and run as a daemon, mongosqld responds to SQL queries, requests for DB information, and schema from the requesting BI tool via the MySQL wire protocol.

mongosqld requires at least one DRDL file to start. You will need to have defined your schema with mongodrdl before starting mongosqld.

SQL Compatibility

MongoDB Connector for BI Version 2.x is compatible with SQL-99 SELECT statements, and uses the MySQL wire protocol.

MongoDB Connector for BI 2.0.2

Released Jan 5, 2017

  • Hides .pem key file passwords in logs.
  • mongodrdl ignores binary fields when generating DRDL files.
  • Supports queries that SELECT variables prefixed with “mongodb”.
  • Fixes incorrect result set from a CROSS JOIN.
  • Supports leading “.” characters on table names.

MongoDB Connector for BI 2.0.1

Released Dec 13, 2016

  • Fixes precedence issue with the NOT operator.
  • Fixes error looking up privilege information in databases with capital letters.
  • Supports LIKE pattern matching in SHOW COLUMNS queries.
  • Improves formatting of aggregation pipeline in logs.
  • Fixes substring function when the combined length of the starting index and substring exceeds the length of the string.