Navigation

Release Notes for MongoDB Connector for BI

On this page

Released March 23, 2021

  • Enables the use of externally sourced values in .yaml configuration files.
  • Adds support for the following fields in .yaml configuration files:

    • trim
    • type
    • digest
    • digest_key
  • Fixes an issue where documents with null values were incorrectly returned from some queries.
  • Fixes an issue where some queries took longer to complete than in previous versions of BI Connector.
  • Fixes an issue where the listDatabases command returned an error when read preference was set to an analytics node in some sharded cluster configurations.
  • Now built with Go 1.5.0.
  • Adds the Amazon Linux 2 ARM 64 distribution to the download center.

Released January 27, 2021

  • Detached signatures are now available for release archives.
  • Fixes an issue that resulted in inefficient translations for in and or expressions.

Released January 6, 2021

  • Now built with Go 1.15.
  • Fixes an issue in which geospatial data did not map correctly.
  • Fixes an issue with regard to DRDL extended character encoding.
  • Fixes an issue in which sharding information failed to load for mixed-case databases.
  • Fixes an issue in which the response to COM_FIELD_LIST was malformed.

Released July 13, 2020

  • Database schemas are now case-sensitive.
  • Queries containing UNION can be fully pushed down.
  • Object-array conflict resolution produces NULL data for conflicting object fields nested in array.

Released February 12, 2020

  • Fixes bug in mapping that did not deep copy pipelines during table post-processing, leading to invalid SQL schemas.

Released January 8, 2020

  • Fixes bug that caused panics on document keys containing characters that need to be escaped, such as " or \usb.

Released January 8, 2020

  • Fixes bug that caused index loading to fail with mixed-case databases.
  • Fixes connectivity bug caused by non-nil interface in custom go-driver dialer.
  • Fixes mongotranslate not including pipelines defined on tables in DRDL file schemas.
  • Fixes bug that caused natural left joins to fail in some cases.

Released November 26, 2019

  • The max_num_global_tables system variable now defaults to 4000, raised from 1000.

Released November 11, 2019

Fixes the following bugs:

  • unix_timestamp returning incorrect results because it did not handle Daylight Savings Time when necessary.
  • DRDL tables incorrectly marshalling to BSON if the aggregation pipeline was empty.

Version 2.13.0 includes the following improvements:

  • Updates an error message for FLUSH SAMPLE authorization that was unclear.
  • Updates schema mapping to skip empty field names and field names containing . characters.
  • Enables self-join optimization for sharded collections.
  • Replaces the max_num_columns_per_table system variable with max_num_fields_per_collection.
  • Enhances performance while sampling collections with many fields by reducing the amount of memory used.
  • Improves ambiguous column behavior in subqueries, GROUP BY and ORDER BY clauses.
  • Adds support for Ubuntu 18.04, SLES 15, and RHEL8.
  • Adds non-string support for regexp.
  • Improves performance for schemas that contain arrays.

Released September 16, 2019

Fixes the following bugs:

  • mongosqld hangs if an election occurred during sampling.
  • Some queries using $lookup fail.
  • Panics occur when connecting with a DRDL file that excludes a child table's _id.
  • Versions 2.10+ install to the wrong directory on Windows.
  • Fixes a bug in the div operator’s in-memory evaluation.
  • mongosqld incorrectly kills long-running queries.
  • Some queries on ObjectId fields do not push down.
  • mongodrdl not parsing replica set seedlists passed to the --host flag.
  • timestampadd fails with out-of-range months.
  • Allows aggregate functions as grouping expressions.
  • datediff fails to push down with polymorphic arguments.
  • Displays incorrect foreign key constraints.
  • Incorrect pushdown and type conversions for date and datetime types that are too large or too small.
  • Aliases for tables in information_schema are ignored.
  • Some queries with aggregate functions fail to push down.
  • Pipeline parsing error for unwind paths with numeric field names.
  • mongodrdl ignores the --gssapiHostName and --gssapiServiceName flags.
  • Allows extra arguments to the current_timestamp and trim functions.
  • str_to_date returns a date instead of a datetime for some non-constant format-string arguments.

This release includes a number of improvements to pushdown. Existing translations were improved and expanded pushdown coverage to more functions and query shapes.

  • Adds support for pushing down correlated subqueries to MongoDB versions 3.6 and later.
  • Adds pushdown support for ln, ascii, user, database, version, connection_id, and constant-valued time/date functions.
  • Adds support for pushing down regexp to the aggregation language.
  • Removes some unnecessary rounding from pushdown translations.
  • Adds support for pushing down LIKE expressions with literal pattern strings to the aggregation language.
  • Improves the efficiency of pipelines generated for EXISTS subqueries.
  • Improves truncate pushdown coverage and updated the translation to use $trunc with MongoDB versions 4.1.9 and later.
  • Adds pushdown support for the char and str_to_date functions.

This release includes a number of improvements made possible by upgrading to the new go driver:

  • Adds new reserved words to the parser: INSERT, INTO, KEY, FULLTEXT, and PRIMARY.

    Going forward, these words must be wrapped in backticks to be used as identifiers in queries.

  • Removes support for altering sampled schema with ALTER TABLE statements.
  • Prior to 2.12, mongosqld supported renaming and removing tables and columns via a modified ALTER TABLE syntax when in standalone schema mode. Any customer currently using this feature is advised to migrate their schema management needs to views, DRDL, or custom schema mode. The variable/config parameter that enabled this feature, enable_table_alterations, is still supported but doesn’t do anything in BI Connector 2.12. It will be removed in a future release.

Released July 8, 2019

Fixes a bug that caused the MySQL schema to include incorrect foreign key constraints.

Fixes a bug that caused incorrect pushdown for some very large or very small date and datetime values.

Fixes a bug that caused aliases to be ignored for tables in the information_schema database.

Released May 16, 2019

  • Fixes a problem that caused the BI Connector to generate unnecessarily long $group key names.
  • Fixes a bug that caused the BI Connector to generate incorrect table pipelines in polymorphic schema mapping mode.
  • Fixes a bug that caused the BI Connector to panic when printing some query plans.
  • Fixes a bug where the BI Connector’s in-memory evaluation of string-to-date conversions did not match the pushdown behavior.
  • Updates the supported Amazon Linux version to match the version supported by MongoDB 4.2.
  • Fixes a bug with Active Directory constrained delegation in the BI Connector’s GSSAPI implementation (linux and macOS).
  • Introduces a --gssapiConstrainedDelegation flag that must be enabled to ensure existing Kerberos configurations are not broken.
  • Fixes a bug that caused Kerberos authentication to fail with a linux mongosqld, MongoDB 4.0, and an Active Directory KDC.
  • Fixes a number of small bugs related to correlated subqueries.
  • Changes some schema-management configuration options.
  • Deprecates schema-modification support via ALTER TABLE, as well as the enable_table_alterations system variable and setParameter

Released Mar 15, 2019

  • Fixes a bug where aggregate functions returned NULL in dual queries with the DUAL keyword omitted.
  • Fixes a bug where some invalid DUAL queries did not return an error.
  • Fixes a bug that prevented pushdown of some joins to derived tables selecting from DUAL.
  • Fixes some correctness bugs in the in-memory evaluation for mysql-mode conversions.
  • Fixes a bug where some trig functions (ascii, acos, and asin) could return incorrect results due to missing type conversions.
  • Fixes a bug where some subquery comparisons could return incorrect results due to missing type conversions.
  • Fixes a bug where we lost precision while converting a Uint64 to a Decimal.
  • Fixes a correctness bug with the INSERT scalar function's pushdown translation.
  • Fixes a bug where unstable sorting caused non-deterministic renaming of colliding collection/field names during schema mapping.
  • Fixes a bug where some valid subquery comparisons returned errors.
  • Fixes a bug where some valid subquery and tuple comparisons returned errors.
  • Fixes a bug where queries using xor returned errors on MongoDB 3.2 because the generated pipelines required features added in MongoDB 3.4.
  • Fixes a bug that caused kerberos authentication to fail with MongoDB 4.0.

This release resolves a number of inconsistencies in the BI Connector's date and timestamp semantics. Resolving these inconsistencies resulted in some changes to user-facing behavior:

  • Removes inconsistent support for partial dates.

    • MySQL can represent partial dates that are missing month or day components. It is impossible to represent these incomplete dates with MongoDB dates. Attempting to correctly manipulate and render incomplete dates was causing a number of bugs and inconsistencies with complete dates.
    • Starting with version 2.10, the BI Connector does not support operations on or expressions that return incomplete dates. Going forward, we cannot provide guarantees about the behavior of operations that attempt to construct incomplete dates.
  • Updates the DATE(a) and TIMESTAMP(a) scalar functions to be equivalent to CAST(a AS DATE), and CAST(a AS DATETIME), respectively.

    • The behavior of the DATE and TIMESTAMP scalar functions will depend on the value of the type_conversion_mode variable. This will change the default behavior for those scalar functions.
    • Queries affected by this change can be rewritten to use a more explicit conversion function (such as str_to_date) that does not depend on type_conversion_mode. If an affected query cannot be rewritten, a user may set type_conversion_mode variable to mysql, which instructs the BI Connector to perform all type conversions with MySQL semantics instead of MongoDB semantics.

      Important

      This is discouraged unless absolutely necessary, as it significantly curtails the ability to push down type conversions across the board.

    • For more information on type conversions and the type_conversion_mode system variable, see Type Conversion Modes.

Released Dec 18, 2018

  • Includes a sample mongosqld configuration file in the release archive.
  • Allows setting commonly used system variables in the configuration file.
  • Optimizes conversions of years to days.
  • Optimizes join performance.
  • General performance improvements.
  • Improves the BI Connector's resiliency to queries against fields that may contain unsupported BSON types.

Released Nov 16, 2018

  • Fixes correctness bugs discovered in polymorphic schema mapping, query cancellation, and IN queries.
  • Fixes a performance regression in a common query pattern.

Released Nov 09, 2018

  • Fixes a bug in the behavior of the max_execution_time system variable.

Released Nov 06, 2018

  • Improvements to the BI Connector’s type system and the way it handles polymorphic data.
  • Users can alter the type of a given column.
  • Users can access “dirty” datasets without query errors.
  • Support for pushdown of the GROUP_CONCAT aggregation function.
  • Non-admin users can set global variables.
  • Improvements in parsing strToDateTime to float.
  • Fixes to how the BI Connector honors readPreference specifications.

Released Oct 02, 2018

  • Collapse associative functions
  • Support the optimize_view_sampling global variable
  • Support the polymorphic_type_conversion_mode session/global variable
  • Improve filter optimization for MongoDB versions 3.6 and later
  • Expose type_conversion_mode in configuration parameters
  • Cache catalog columns for quicker lookup
  • Fix incorrect semantics for null safe equality operator ( <=> )
  • The || operator returns incorrect type (and value) when operating on two strings
  • Add DRDL type validation
  • Support CAST to unsigned
  • Fix WHERE clause removed in certain cross-join queries
  • Fix some incorrect left self-join results when dynamic schemas are used
  • Support turning off view optimization with $match
  • ODBC driver on macOS should install in versioned directories
  • Fix ODBC driver installer issue on macOS that caused unicode driver to be reported as ANSI

Released version 1.2.0 of the MongoDB ODBC Driver.

Released Sep 21, 2018

  • Resolved an issue caused by how scalar functions handle string values.

We encourage Tableau users running BIC 2.6.0 to upgrade to 2.6.1.

Released Sep 04, 2018

  • Added EXPLAIN() support.
  • Push down type conversions to MongoDB servers running version 4.0 or later.
  • Improved the handling of polymorphic data by updating the BI Connector's type conflict resolution system.
  • Added Common Table Expression (CTE) support.
  • Improved UNION performance.
  • Added the SCRAM-SHA-256 mechanism to the following:

    • mongosqld command line options and configuration file settings:

      Command Line Option
      Configuration File Setting
    • mongodrdl --authenticationMechanism command line option.
    • MongoDB ODBC driver
    • C Authentication Plugin
    • Java Authentication Plugin
  • Released the MongoDB ODBC driver for Linux.
  • Added the mongosqld --prejoin command line option for combining array data and non-array data into a single table.

Users can use EXPLAIN to return details of aggregation pipeline used to generate query results.

The BI Connector now supports queries that include Common Table Expressions.

Released May 23, 2018

Windows Clients may now use kerberos to authenticate against the BI Connector.

This resolves the issue where queries would fail due to NULL values being ignored during sampling. Now, NULL values will be cast as SQL type VARCHAR.

BI Connector will now push down SIN, COS, and TAN SQL functions. This will improve performance for queries using these functions.

Released Mar 09, 2018

We've released version 2.4.1 of the MongoDB Connector for BI for production deployment. This release contains minor bug fixes for users of the BI Connector and is a recommended upgrade for all 2.4 users.

Released Feb 21, 2018

MongoDB Connector for BI version 2.4 improves more expressive queries. New features include:

BI Connector 2.4 improves support for aggregate expressions including math, date, and string functions.

With version 2.4, mongosqld has improved support for subqueries and non-equijoins.

Released Nov 10, 2017

  • Fixes minor incompatibility with MongoDB 3.2.

Released Nov 08, 2017

  • Simplified and centralized Schema Management.
  • Supports log rotation.
  • Standardized logging in the BI Connector.
  • Push LTRIM, RTRIM and TRIM functions down to MongoDB.

MongoDB Connector for BI version 2.3 improves enterprise deployment support. The new features include:

In BI Connector 2.3, mongosqld includes the schema mapping functionality previously found in mongodrdl.

With version 2.3, you can rotate the mongosqld logs. To rotate the logs, issue a command or restart mongosqld.

Released Aug 29, 2017

  • Fixes lack of support for Read Preference tags which are used when targeting reads to specific members of a replica set.

Released Jul 31, 2017

  • Supports authenticating directly against MongoDB using the new C and JDBC authentication plugins. These plugins support SCRAM-SHA-1 and PLAIN mechanisms and remove the SSL requirement for authentication. The authentication plugins can be found on GitHub:

  • Supports configuration files for mongosqld.
  • Supports installing and running mongosqld as a Windows system service.
  • Provides a Windows MSI installer for the MongoDB Connector for BI.
  • Provides new command line arguments for default auth mechanism and database:

    • Authentication Options:

      --defaultAuthDatabase
      default user source (default of admin)
      --defaultAuthMechanism arg
      default authentication mechanism (default of SCRAM-SHA-1)
  • Provides a new startup option to specify default VARCHAR length
  • Provides mongosqld status metrics with SHOW STATUS

    • Reports the following metrics reported:
      • Connections
      • Queries
      • BytesSent
      • Threads Connected
  • Can bind mongosqld to multiple IP addresses
  • Provides a new runtime options to limit memory usage of GROUP, SORT, and JOIN stages

MongoDB Connector for BI Version 2.0 introduced passthrough authentication, in which clients authenticate as a user defined in MongoDB, without the need to create a separate user within the connector itself.

MongoDB Connector for BI Version 2.2 further increases ease of use with a set of authentication plugins which can authenticate users through the connector without the need for SSL certificates. These plugins perform user authentication via the SASL framework and support the SCRAM-SHA-1 and PLAIN authentication mechanisms. We currently provide authentication plugins in the following languages:

  • Java - for use with MySQL’s Connector/J driver
  • C - for use with the MySQL ODBC driver and the MySQL Shell
  • Provides substantial performance improvements for some common JOIN scenarios.
  • Pushes $match stages earlier in the aggregation pipeline to improve to index utilization.
  • Pushes down more SQL functions and operators to the aggregation pipeline.
  • Supports additional SQL language features, such as USE and SHOW CREATE TABLE statements, STRAIGHT_JOIN, the unary plus (+) operator, and the INTERVAL keyword in date arithmetic.
Note

mongosqld no longer offers the --fork option.

To run mongosqld in the background, use the nohup command:

nohup mongosqld [options] &

Released Nov 29, 2016

The MongoDB Connector for BI delivers significant performance and scalability improvements by moving as much execution as possible to the underlying MongoDB instance. Pushing execution down to the server reduces network latency and reduces bandwidth consumption.

This approach also reduces memory and CPU consumption in the MongoDB Connector for BI itself.

The user authentication process is now passed through the MongoDB Connector for BI to the underlying MongoDB instance. Users now authenticate as a user already created within MongoDB.

The MongoDB Connector for BI package is now trimmed down from four separate components to two:

mongodrdl
This tool connects to a MongoDB instance and generates a Document-Relational Definition Language (DRDL) file, which maps a given collection shape to relational schema.
mongosqld

Once installed and run as a daemon, mongosqld responds to SQL queries, requests for DB information, and schema from the requesting BI tool via the MySQL wire protocol.

mongosqld requires at least one DRDL file to start. You will need to have defined your schema with mongodrdl before starting mongosqld.

MongoDB Connector for BI Version 2.x is compatible with SQL-99 SELECT statements, and uses the MySQL wire protocol.

Tip
See also:

Released Jan 5, 2017

  • Hides .pem key file passwords in logs.
  • mongodrdl ignores binary fields when generating DRDL files.
  • Supports queries that SELECT variables prefixed with "mongodb".
  • Fixes incorrect result set from a CROSS JOIN.
  • Supports leading "." characters on table names.

Released Dec 13, 2016

  • Fixes precedence issue with the NOT operator.
  • Fixes error looking up privilege information in databases with capital letters.
  • Supports LIKE pattern matching in SHOW COLUMNS queries.
  • Improves formatting of aggregation pipeline in logs.
  • Fixes substring function when the combined length of the starting index and substring exceeds the length of the string.
Give Feedback

On this page