load_meta_data
get_objects_by_folder
def get_objects_by_folder(
tablefolder: str,
config_manager: ConfigManager) -> list[TableConfig] | None
Loads metadata configurations from a specified folder or file and returns a list of TableConfig objects. If a single file is specified, metadata will be loaded only from that file. Otherwise, the function traverses the specified folder and loads metadata from all YAML files.
Arguments:
tablefolder
str - The path to the folder or file containing YAML metadata files. If it starts with "Files/", it will be resolved to the default lakehouse path.config_manager
ConfigManager - An instance of ConfigManager used to initialize and validate configurations.
Returns:
List[TableConfig]
- A list ofTableConfig
dataclass instances created from the loaded YAML metadata.
Raises:
Exception
- Ifconfig_manager
is not initialized or if an error occurs during metadata loading andconfig_manager.stop_at_error
is set to True.
get_model_by_file
def get_model_by_file(modelfile: str, config_manager: ConfigManager) -> Model
Loads a Model object by reading a given YAML file. Ensures that the configuration manager is initialized and sets up logging for the operation.
If the input file path starts with a specific prefix, it is converted to an absolute path. The function validates the file type to ensure it is a YAML file and parses its content to create a Model instance.
Arguments:
modelfile
- The path to the metadata YAML file. It can be relative or absolute. If it starts with 'Files/', it is converted to the absolute path in the lakehouse directory.config_manager
- An instance of ConfigManager that is responsible for initialization and managing configurations.
Returns:
Model
- The Model object created from the provided YAML file.
Raises:
Exception
- If the configuration manager is not initialized.Exception
- If the provided file is not a YAML file.Exception
- If the stop-at-error policy is enabled in the configuration manager and an error occurs during the process.