`mongodrdl`¶

On this page

Description
Syntax
Default Minimum TLS Version
Commands
Options
Usage Examples
MongoDB Atlas Example

Description¶

mongodrdl¶

Note

The MongoDB Connector for BI and associated utilities are compatible only with MongoDB server version 3.2 or greater.

mongodrdl is a relational schema management tool for the MongoDB Connector for BI. The mongodrdl binary can:

Produce a schema based on the contents of one or more MongoDB collections and write them out into .drdl files used by mongosqld.
Upload a .drdl file’s schema to the MongoDB deployment your BI Connector is reading from.
Download stored schemas from your MongoDB deployment.
Delete stored schemas from your MongoDB deployment.
Name stored schemas in your MongoDB deployment for easier management.

You can edit a .drdl file manually to customize the BI Connector’s schema. To learn more about the .drdl format and manually editing .drdl files, see Load a Schema from a DRDL File.

To learn more about all BI Connector schema management options, see Map Relational Schemas to MongoDB.

Important

To apply any changes you make to your DRDL files, you must restart mongosqld.

Syntax¶

mongodrdl uses the following syntax:

copy

mongodrdl [<command>] [<options...>]

Default Minimum TLS Version¶

New in version 2.6.

The default minimum TLS version is 1.1 for outgoing connections from mongodrdl to MongoDB.

You can adjust this setting with the --minimumTLSVersion option.

Commands¶

New in version 2.11.

The behavior of mongodrdl is determined by one of the following commands. If no command is specified, mongodrdl samples the specified namespace.

sample¶

Samples the namespaces specified by the --db, optional --collection, and other command line options to output a .drdl file. The sample command encapsulates all the mongodrdl behavior prior to version 2.11.

copy

mongodrdl sample --db <db-name> --collection <collection-name> --out <filename>.drdl

list-schema-ids¶

Lists the string representation of the ObjectId of each stored schema and the date it was created.

list-schema-ids requires the following option:

Name	Description
`--schemaSource`	Required. Specifies the database where the schema information is stored.

For example, the following command lists the schema ids in the schemas database:

copy

mongodrdl list-schema-ids --schemaSource schemas

The output resembles the following:

5d72ad695c23a9e3e26e3c85 2019-09-06T15:03:05.556Z
5d72adea5c23a9e44882a1ad 2019-09-06T15:05:14.349Z

list-schema-names¶

Lists the name and string representation of the ObjectId for each stored schema that has a name.

Stored schemas are unnamed by default. You can refer to a named schema by using the --name option with the download and delete commands. Use the name-schema command to name or rename a schema.

list-schema-names requires the following option:

Name	Description
`--schemaSource`	Required. Specifies the database where the schema information is stored.

For example, the following command returns the name and ObjectId for each named schema in the schemas database:

copy

mongodrdl list-schema-names --schemaSource schemas

The output resembles the following:

movies 5d72ad695c23a9e3e26e3c85
theaters 5d72adea5c23a9e44882a1ad

upload¶

Uploads the schema from the specified .drdl file to the database specified by the --schemaSource option. Schemas are uploaded to the schemas collection.

upload requires the following options:

Name	Description
`--schemaSource`	Required. Specifies the database where the schema information is stored.
`--drdl`	Required. Path to the `.drdl` file to upload.

For example, the following command uploads the schema in the movies.drdl file to the schemas database:

copy

mongodrdl upload --schemaSource schemas --drdl ./movies.drdl

The string representation of the ObjectId of the uploaded schema is returned:

5d793f3f6a26a3ce66c304ea

To use the newly uploaded schema, restart mongosqld with the schema database specified by the --schemaSource option and the --schemaMode set to custom:

copy

mongosqld --schemaMode custom --schemaSource <schema-db>

download¶

Downloads a schema specified by its name or string representation of the ObjectId and prints it to the console.

download accepts the following options:

Name Description

--schemaSource Required. Specifies the database where the schema information is stored.

--name

The name of the schema. To learn more about naming schemas, see the name-schema command.

Required if --schema is not specified.

--schema

The string representation of the ObjectId of the schema.

Required if --name is not specified.

For example, the following command prints the movies schema to the console:

copy

mongodrdl download --schemaSource schemas --name movies

To store the schema in a .drdl file, direct the output to the desired .drdl file:

copy

mongodrdl download --schemaSource schemas --name movies > ./movies.drdl

delete¶

Deletes a schema specified by its name or string representation of the ObjectId.

delete accepts the following options:

Name Description

--schemaSource Required. Specifies the database where the schema information is stored.

--name

The name of the schema. To learn more about naming schemas, see the name-schema command.

Required if --schema is not specified.

--schema

The string representation of the ObjectId of the schema.

Required if --name is not specified.

For example, the following command deletes the schema with id 5d7941dc6a26a3d0fc397284 in the schemas database:

copy

mongodrdl delete --schemaSource schemas --schema 5d7941dc6a26a3d0fc397284

name-schema¶

Creates a new name for a schema or updates a schema’s existing name.

Stored schemas are unnamed by default. You can refer to a named schema by using the --name option with the download and delete commands. Use the name-schema command to name or rename a schema.

name-schema requires the following options:

Name	Description
`--schemaSource`	Required. Specifies the database where the schema information is stored.
`--name`	Required. The new name of the schema.
`--schema`	Required The string representation of the ObjectId of the schema.

For example, the following command names the 5d72adea5c23a9e44882a1ad schema in the schemas database as movies:

copy

mongodrdl name-schema --name movies --schemaSource schemas --schema 5d72adea5c23a9e44882a1ad

Options¶

Core Options¶

--help¶: Returns information on the options and use of mongodrdl.

--version¶: Returns the mongodrdl release number.

--verbose <level>, -v <level>¶: Specifies that mongodrdl should provide more detailed log output. Include multiple times for more verbosity (e.g. -vvvvv), or specify a numeric value (e.g. --verbose=5).

--quiet¶: Hides all log output.

--uri <mongodb-uri>¶

New in version 2.12.

MongoDB URI connection string.

Important

The following command-line options cannot be used in conjunction with --uri option:

--host
--port
--db
--username
--password (if the URI connection string also includes the password)
--authenticationDatabase
--authenticationMechanism

Instead, specify these options as part of your --uri connection string.

--host <hostname><:port>, -h <hostname><:port>¶

Default: localhost:27017

Specifies a resolvable hostname for the mongod to which to connect. By default, the mongodrdl attempts to connect to a MongoDB instance running on the localhost on port number 27017.

To connect to a replica set, specify the replSetName and a seed list of set members, as in the following:

copy

<replSetName>/<hostname1><:port>,<hostname2><:port>,<...>

You can always connect directly to a single MongoDB instance by specifying the host and port number directly.

--port <port>¶

Default: 27017

Specifies the TCP port on which the MongoDB instance listens for client connections.

--db <database>, -d <database>¶: Specifies a database from which to generate a .drdl schema file.

--collection <collection>, -c <collection>¶: Specifies a collection from which to generate a .drdl schema file. If you do not specify a collection, this option will use all collections in the specified database or instance.

--customFilterField <name>, -f <name>¶: Specifies the field name to add for a custom MongoDB filter. See Custom Filters for more details.

--out <path>, -o <path>¶

Default: Standard out.

Specifies the path where mongodrdl will write the schema file. To send the schema to standard output, specify “-” instead of a path.

--sampleSize <size>, -s <size>¶

Default: 1000

Specifies the number of documents to sample when generating the collection’s schema.

--uuidSubtype3Encoding <old|csharp|java>, -b <old|csharp|java>¶

Specify the encoding used to generate UUID binary subtype 3. Choose one of the following values:

old: Old BSON binary subtype representation
csharp: The C#/.NET legacy UUID representation
java: The Java legacy UUID representation

--preJoined¶: Generate unwound tables including parent columns, resulting in a “pre-joined” table.

--schemaSource <db-name>¶: New in version 2.11.

Specifies the database where the schema information is stored. This option is not used by the sample command.

TLS/SSL Options¶

--ssl¶

Default: False

Instructs mongodrdl to use TLS/SSL when connecting to a MongoDB instance.

--sslCAFile <filename>¶: Specifies the MongoDB instance’s .pem file containing the root certificate chain from the Certificate Authority. Specify the file name of the .pem file using relative or absolute paths.

Warning

For SSL connections (--ssl) to mongod and mongos, if the mongodrdl runs without the --sslCAFile, mongodrdl will not attempt to validate the server certificates. This creates a vulnerability to expired mongod and mongos certificates as well as to foreign processes posing as valid mongod or mongos instances. Ensure that you always specify the CA file to validate the server certificates in cases where intrusion is a possibility.

--sslPEMKeyFile <filename>¶

Specifies the .pem file containing both the TLS/SSL certificate and key for mongosqld to use when connecting to MongoDB. You can specify the file name of the .pem file using either using a relative or absolute path.

This option is required when using the --ssl option to connect to a mongod or mongos that has CAFile enabled without allowConnectionsWithoutCertificates.

--sslPEMKeyPassword <password>¶: Specifies the path to a file containing the certificate and private key for connecting to MongoDB.

--sslCRLFile <filename>¶: Specifies the .pem file that contains the Certificate Revocation List. Specify the file name of the .pem file using relative or absolute paths.

--sslAllowInvalidCertificates¶: Permits the MongoDB instance to present an invalid server SSL/TLS certificate. When using the allowInvalidCertificates setting, MongoDB logs the use of the invalid certificate as a warning.

--sslAllowInvalidHostnames¶: Disables the validation of the hostnames in TLS/SSL certificates. Allows mongodrdl to connect to MongoDB instances if the hostname their certificates do not match the specified hostname.

--sslFIPSMode¶: Directs the mongodrdl to use the FIPS mode of the installed OpenSSL library. Your system must have a FIPS compliant OpenSSL library to use the --sslFIPSMode option.

--minimumTLSVersion <TLS1_0|TLS1_1|TLS1_2>¶

Default: TLS1_1

Specifies the minimum required TLS version for mongodrdl to connect to a mongod or mongos instance. The default value TLS1_1 corresponds to TLS version 1.1.

Authentication Options¶

--username <username>, -u <username>¶: Specifies a username with which to authenticate to a MongoDB database that uses authentication. Use in conjunction with the --password and --authenticationDatabase options.

--password <password>, -p <password>¶: Specifies a password with which to authenticate to a MongoDB database that uses authentication. Use in conjunction with the --username and --authenticationDatabase options.

--authenticationDatabase <dbname>¶: Specifies the database in which the user is created. See Authentication Database.

--authenticationMechanism <name>¶

Default: SCRAM-SHA-1

Specifies the authentication mechanism the mongodrdl instance uses to authenticate to the mongod or mongos.

Value	Description
SCRAM-SHA-1	RFC 5802 standard Salted Challenge Response Authentication Mechanism using the SHA1 hash function.
SCRAM-SHA-256	New in version 2.6: RFC 7677 standard Salted Challenge Response Authentication Mechanism using the SHA2 hash function.
PLAIN (LDAP SASL)	External authentication using LDAP. You can also use `PLAIN` for authenticating in-database users. `PLAIN` transmits passwords in plain text. This mechanism is available only in MongoDB Enterprise.
GSSAPI (Kerberos)	External authentication using Kerberos. This mechanism is available only in MongoDB Enterprise.

--gssapiServiceName <service-name>¶

Default: mongodb

Service name to use when authenticating using GSSAPI/Kerberos.

--gssapiHostName <hostname>¶

Default: remote server’s address

Hostname to use when authenticating using GSSAPI/Kerberos.

Custom Filters¶

BI Connector uses a MongoDB aggregation pipeline to perform queries against a MongoDB database. You can add a special field to your schema to pass a custom MongoDB $match query string to your MongoDB instance.

Use the --customFilterField option with mongodrdl to name a custom filter field. This field passes a native MongoDB $match stage to use as the first stage of the aggregation pipeline, which filters any results returned from MongoDB. The BI Connector applies this stage before any additional stages pushed down from SQL.

The query can refer to any field in the collection, even if the fields are not exposed in the relational schema.

For an example of custom filter usage, see Custom Filter Example.

Usage Examples¶

Schema Generation Example¶

Given documents of the following shape in the collection abc in the database test:

copy

{
    "_id": ObjectId(),
    "close": 7.45,
    "detail": { "a": 2, "b": 3 }
}

Run mongodrdl to generate a schema based on this collection:

copy

mongodrdl -d test -c abc -o schema.drdl

The generated schema file (schema.drdl) looks similar to the following:

copy

schema:
- db: test
  tables:
  - table: abc
    collection: abc
    pipeline: []
    columns:
    - Name: _id
      MongoType: bson.ObjectId
      SqlName: _id
      SqlType: varchar
    - Name: close
      MongoType: float64
      SqlName: close
      SqlType: numeric
    - Name: detail.a
      MongoType: float64
      SqlName: detail.a
      SqlType: numeric
    - Name: detail.b
      MongoType: float64
      SqlName: detail.b
      SqlType: numeric

Custom Filter Example¶

To use this field, specify the --customFilterField flag with the name you want this field to have:

copy

mongodrdl [ other options ] --customFilterField _MONGOFILTER -o schema.drdl

Your DRDL file schema.drdl will include the following field in every generated table:

copy

- Name: _MONGOFILTER
  MongoType: mongo.Filter
  SqlName: _MONGOFILTER
  SqlType: varchar

To add the special MongoDB query stage to your standard SQL, use the following SQL syntax:

copy

SELECT <normal>
  FROM <tablename>
  WHERE <normal conditions> AND
      _MONGOFILTER='{ <json string that represents query to use> }'

SELECT name,age
  FROM users
  WHERE active='t' AND
      _MONGOFILTER='{"addr":{"$elemMatch":{"city":"Springfield","state":"CA"}}}'

The BI Connector will translate the above SQL into the following MongoDB aggregation expression:

copy

db.users.aggregate([
   {$match:{"addr":{"$elemMatch":{"city":"Springfield","state":"CA"}}},
   {$match:{"active":true}},
   {$project:{"name":1, "age":1}}
]);

You can use this custom filter in any business intelligence tool by filtering on your special field and providing the value to match as a single quoted string representing valid JSON. All quotes inside the JSON must be double quotes.

Authentication Example¶

If your MongoDB server is running with authentication enabled, you must specify a username, password, and authentication database as options to your mongodrdl command.

To create a .drdl file from a mongod instance with authentication enabled, use the following command format:

copy

mongodrdl --host myhost.example.net:27017 \
          --username dbUser \
          --password myPassword \
          --db reports \
          --collection FY2017 \
          --authenticationDatabase admin \
          --out schema.drdl

MongoDB Atlas Example¶

MongoDB Atlas is a cloud service for running, monitoring, and maintaining MongoDB deployments, including the provisioning of dedicated servers for MongoDB instances.

Note

MongoDB Atlas now offers a hosted BI Connector and does not require a .drdl file. For more information on the Atlas-hosted BI Connector, see Connect via BI Connector for Atlas.

If you are running the BI Connector on premises and wish to create a .drdl file from an Atlas database, use the following command format:

copy

mongodrdl --host <domain0>.mongodb.net:27017,<domain1>.mongodb.net:27017,... \
          --username <username> \
          --password <password> \
          --ssl \
          --authenticationDatabase admin \
          --db <database> \
          --out schema.drdl

← mongosqld mongotranslate →

mongodrdl¶

Description¶

Syntax¶

Default Minimum TLS Version¶

Commands¶

Options¶

Core Options¶

TLS/SSL Options¶

Authentication Options¶

Custom Filters¶

Usage Examples¶

Schema Generation Example¶

Custom Filter Example¶

Authentication Example¶

MongoDB Atlas Example¶

`mongodrdl`¶