Manage Namespace Metadata Catalog for Wildcard Collections¶

The Atlas Data Lake namespace catalog contains namespace metadata (such as databases, collections, and partitions) for each wildcard (*) collection in each data store in the Data Lake storage configuration. For commands and queries on wildcard collections, Atlas Data Lake returns the data from the catalog or returns an error if the data store does not exist.

When objects that are included in a wildcard collection specification are modified in your S3 bucket, the catalog must be updated for you to see the changes when you run list commands against wildcard collections. By default, the catalog is refreshed every hour for all active Data Lakes. In addition, the namespace metadata catalog is refreshed in the background when you:

• Update the storage configuration.
• Run the show collections command.
• Query a collection.
• Run the updateCatalog command.

You can determine when the catalog was last updated by using the catalogInfo command and can manually update the catalog by using the updateCatalog command. You must have the atlasAdmin role to update the catalog.