Manage Namespace Metadata Catalog for Wildcard Collections¶
The Atlas Data Lake namespace catalog contains namespace metadata
(such as databases, collections, and partitions) for each wildcard (*
)
collection in each data store in the Data Lake storage
configuration. For commands and queries on wildcard collections,
Atlas Data Lake returns the data from the catalog or returns an error if the
data store does not exist.
When objects that are included in a wildcard collection specification are
modified in your S3 bucket, the catalog must be updated for you to see
the changes when you run list
commands against wildcard collections.
By default, the catalog is refreshed every hour for all active
Data Lake s. In addition, the namespace metadata catalog is
refreshed in the background when you:
- Update the storage configuration.
- Run the
show collections
command. - Query a collection.
- Run the
updateCatalog
command.
You can determine when the catalog was last updated by using the
catalogInfo
command and can manually update the catalog
by using the updateCatalog
command. You must have the
atlasAdmin role
to update the catalog.