Manage Namespace Metadata Catalog for Wildcard Collections¶
Beta
The Atlas Data Lake is available as a Beta feature. The product and the corresponding documentation may change at any time during the Beta stage. For support, see Atlas Support.
The Atlas Data Lake namespace catalog contains namespace metadata
(such as databases, collections, and partitions) for each wildcard (*)
collection in each data store in the Data Lake storage
configuration. For commands and queries on wildcard collections,
Atlas Data Lake returns the data from the catalog or returns an error if the
data store does not exist.
When objects that are included in a wildcard collection specification are
modified in your S3 bucket, the catalog must be updated for you to see
the changes when you run list commands against wildcard collections.
By default, the catalog is refreshed every hour for all active
Data Lake s. In addition, the namespace metadata catalog is
refreshed in the background when you:
- Update the storage configuration.
- Run the
show collectionscommand. - Query a collection.
- Run the
updateCatalogcommand.
You can determine when the catalog was last updated by using the
catalogInfo command and can manually update the catalog
by using the updateCatalog command. You must have the
atlasAdmin role
to update the catalog.