input.esm_catalog_builder#

Classes

CatalogBuilder(catalog_id[, datasets_info])

class CatalogBuilder(catalog_id, datasets_info: dict | str = None)[source]#

_process_dataset_for_catalog(dataset_name, dataset_info)[source]#

Process all files in a dataset and extract metadata for each file.

Given a dataset name and its information, this function parses every file in the dataset, returning the metadata parsed from the file name and the dataset level metadata.

Parameters:

dataset_name (str) – The name of the dataset to process.
dataset_info (dict) – The dataset information containing the root directory, regex pattern, and metadata.

Returns:

A list of dictionaries containing metadata for each file in the dataset.

Return type:

list

_validate_dataset_info()[source]#: Validate the dataset information to ensure all required identifiers are present.

add_dataset(dataset_name, dataset_info)[source]#

Update the dataset information for a specific dataset.

Parameters:

dataset_name (str) – The name of the dataset to update.
dataset_info (dict) – The new dataset information to update.

create_df()[source]#: Create a catalog by scanning dataset paths and extracting metadata.