Partition Attribute Types¶
Supported Partition Attribute Types¶
The following table lists the supported data types for partition
attributes, filename, and
path
example for each data type:
Key | Data Type | Example | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
string | Parses the filename as a string. | filename: path: In the preceding example, Data Lake interprets Tip See also: | ||||||||||||||||||
int | Parses the filename as an integer. | filename: path: In the preceding example, Data Lake interprets Tip See also: | ||||||||||||||||||
isodate | Parses the filename in RFC 3339 format as an ISO-8601 format date. | filename: path: In the preceding example, Data Lake interprets
| ||||||||||||||||||
epoch_secs | Parses the filename as a Unix timestamp in seconds. | filename: path: In the preceding example, Data Lake interprets Tip See also: | ||||||||||||||||||
epoch_millis | Parses the filename as a Unix timestamp in milliseconds. | filename: path: In the preceding example, Data Lake interprets Tip See also: | ||||||||||||||||||
objectid | Parses the filename as an ObjectId. | filename: path: In the preceding example, Data Lake interprets | ||||||||||||||||||
uuid | Parses the filename as a UUID of binary subtype 4. | filename: path: In the preceding example, Data Lake interprets |
Atlas Data Lake supports the Package Syntax for regular expressions in the path to the filename.
Supported Conversions¶
Atlas Data Lake converts the partition attributes to BSON types when parsing the
path
to the filename. Later writes of data to S3 must use the BSON types
after converting them to string. The following table shows:
- The partition attribute types and the BSON types to which Data Lake converts them.
- The BSON data type to convert to a string for later writes to S3 .
Partition Attribute Type | Parsed BSON Type | Source BSON Type |
---|---|---|
string |
|
|
int |
|
|
isodate |
|
|
objectid |
|
|
uuid |
|
|