Custom extensions
Every data platform has its unique characteristics. While many situations are similar, each company has its own specific requirements. Sometimes, it is necessary to perform actions that do not fit the standard workflow. In these scenarios, it is helpful to know that custom actions can be added to the flow.
The most logical place to implement these custom actions is in a notebook.
Between every layer, there is an option to run a custom notebook.
Pre notebook and post notebook
To run a specific notebook before loading data into a layer, define a Pre notebook on the object yaml file. In this case before running bronze:
Table:
Connection: exa-example
SourceTable: example.json
DataPlatformObjectname: example
PreBronzeNotebook:
Notebook: Pull_example_file
Param001: SomeValue
When the action to load bronze data is activated and if a prebronzenotebook is defined, this notebook will be run. The following parameters are supplied to it automatically:
sourcetablesourcefilterbronzefolderkeyvaultobject_yaml_file— the path to this object's YAML file (not its contents). Use it to read the full object definition inside the notebook (see Reading the object).
A few internal context parameters are also passed (notebook_type, log_file_path, base_batch_id); you can ignore these unless you are integrating with EasyFabric's logging.
Optionally it is possible to supply 3 additional parameters (Param001, Param002, Param003) on the notebook node in the YAML. These arrive lower-cased as param001–param003. In this example there is one parameter Param001 supplied with a value of SomeValue, so the notebook receives param001="SomeValue".
The notebook that is called should have a parameter cell with the following:
sourcetable=""
sourcefilter=""
bronzefolder=""
keyvault=""
object_yaml_file=""
param001=""
param002=""
param003=""
Works equally as the pre notebook, but is run after the loading has finished.
The same contract for every hook
These parameters are not specific to bronze or to the pre position. Every layer hook receives the identical set, so a notebook written for one slot works in any of them. The hooks are wired on the object YAML under:
| Around bronze | Around silver |
|---|---|
PreBronzeNotebook | PreSilverNotebook |
MidBronzeNotebook | MidSilverNotebook |
PostBronzeNotebook | PostSilverNotebook |
All six pass the same standard parameters (including object_yaml_file) plus their own Param001–Param003.
Reading the object inside the notebook
Because object_yaml_file is a path, the notebook reads the object definition itself. The canonical way is load_meta_data.get_object_by_file, which returns a fully-initialised TableConfig:
from easyfabric import load_meta_data
# object_yaml_file is supplied by EasyFabric; guard for it so the cell is
# still runnable on its own during development.
if "object_yaml_file" in locals():
obj = load_meta_data.get_object_by_file(object_yaml_file) # -> TableConfig
target = f"Files/{obj.bronzefolder}/{obj.sourcetable}"
get_object_by_file initialises the ConfigManager and applies the normal config-aware defaults. If you only want a raw parse with no config context, use TableConfig.from_yaml_file(object_yaml_file) instead. Do not use get_tableconfig_by_inputfile here — that resolves a data file back to its config by bronzefolder prefix, which is the opposite direction.
The notebooks above run around an existing file or database source. When there is no such source — the data lives behind an API like Microsoft Graph / Entra and the notebook itself fetches it — see Connecting a source to bronze.