YAML Properties Reference

This page provides a comprehensive overview of all YAML properties used across the EasyFabricCore system. It specifically identifies which properties are utilized by the major system components.

Component Overview

• API: Web API that receives JSON requests (converted from YAML) to generate Fabric objects.
• Dataplatform: Desktop application (Generator.exe) that reads YAML files to generate notebooks, tables, and configurations.
• Wheel Package (easyfabric-py): Python library used at runtime in Fabric notebooks to load and process data.

---

Configuration Properties

Defined in the main environment configuration file (e.g., dp.dev.yaml).

Property	API	Dataplatform	Wheel Package	Description
BronzeConnectionPrefix	✅	✅	✅	Use prefix from ConnectionPrefix for bronze objects (default=true)
BronzeKeepSourceColumnNames	✅	✅	—	Use source names even when overridden on bronze (default=false)
BronzeObjectPrefix	✅	✅	✅	Prefix for bronze objects (default='')
BronzeTableSettings	✅	✅	—	Table settings for Bronze layer (Delta properties)
ColumnNamesToLower	✅	✅	—	Generate all column names to lower (default=true)
ColumnNamesAutoFix	✅	✅	—	Auto fix columns with only numbers by adding 'z' prefix (default=true)
DefaultSeparator	✅	✅	✅	Separator used for object names (default='_')
DimBusinesskeyPrefix	✅	✅	—	Prefix of businesskey column for dimensions (default='BK_')
DimObjectPrefix	✅	✅	—	Prefix used for dimensions (default='D')
DimSurrogatekeyPrefix	✅	✅	—	Prefix used for dimension surrogate keys (default='SK_')
FactObjectPrefix	✅	✅	—	Prefix used for fact objects (default='F')
HistoryTablePostfix	✅	✅	✅	Postfix used for objects with history (default='')
IsBronzeStringTypes	✅	✅	✅	Create fields in bronze layer with datatype string (default=true)
Lakehouses	✅	✅	✅	List of lakehouses used in medallion layer
ObjectPartitionColumn	✅	✅	—	Name of partition column (default='SYSTEMPARTITION')
ObjectSourceTagColumn	✅	✅	—	Name of system source tag column (default='SYSTEMSOURCETAG')
RemoveNewLineFromScripts	✅	✅	—	Remove newline from generated scripts (default=true)
SchemaHistory	✅	✅	✅	Schema name for history tables (default='his')
SilverConnectionPrefix	✅	✅	✅	Use connection prefix for object names on silver (default=true)
SilverConvertedColumn	✅	✅	—	Create Silver column with converted value (default=true)
SilverConvertedPostfix	✅	✅	✅	Postfix for converted column in silver (default='_convert')
SilverInitialColumn	✅	✅	—	Create Silver column with original value (default=true)
SilverObjectPrefix	✅	✅	—	Prefix for silver objects (default='')
SilverPrimarykeySeparator	✅	✅	—	Separator for SYSTEMPRIMARYKEY field (default='
SilverTableSettings	✅	✅	—	Table settings for Silver layer (Delta properties)
SurrogateKeyUnknownValue	✅	✅	—	Default value for surrogate key when no match (default='-1')
TransformationBusinessKeysNotNull	✅	✅	—	Business keys in transformation objects mandatory (default=false)
TransformationObjectPrefix	✅	✅	—	Prefix used for transformation objects (default='T')
DefaultEncodingHintKeyColumns	✅	✅	—	Default encoding hint for key columns (default='Value')
DefaultEncodingHintRegularColumns	✅	✅	—	Default encoding hint for regular columns (default='Default')

---

Connection Properties

Defined in connection YAML files (e.g., adv-advworks.yaml).

Property	API	Dataplatform	Wheel Package	Description
ConnectionName	✅	✅	✅	Name of the connection
ConnectionPrefix	✅	✅	✅	Prefix used when connection prefix is enabled
ConnectionType	—	—	✅	Type of connection (e.g., fabricfiles, sqlserver)
BronzeFolder	—	—	✅	Folder path for bronze files
BronzeKeyViolationAction	—	—	✅	Action on key violations (raise/keepone/remove)
BronzeLoadOrder	—	—	✅	Load order for bronze notebooks
BronzeLoadSkip	—	—	✅	Skip bronze loading (default=false)
BronzeNotebookConcurrency	—	—	✅	Concurrency for bronze notebooks (default=0)
BronzeTableSettings	✅	✅	—	Table settings for Bronze at connection level
Charset	—	—	✅	Character set for file reading
Container	—	—	✅	Storage container name
Dateformat	—	—	✅	Default date format for parsing
Delimiter	—	—	✅	CSV delimiter (default=',')
Escape	—	—	✅	Escape character for CSV
FileExtension	—	—	✅	File extension filter
FileType	—	—	✅	Type of file (csv, json, xml, parquet, etc.)
Header	—	—	✅	CSV has header row (default=true)
IgnoreNullKey	—	—	✅	Ignore records with null keys (default=true)
InferSchema	—	—	✅	Infer schema from data (default=false)
IsBronzeStringTypes	✅	✅	✅	Override config setting for this connection
IsTriggerable	—	—	✅	Connection can be triggered (default=true)
JsonPath	—	—	✅	JSON path for extracting data
KeyvaultSecretConnectionString	—	—	✅	Keyvault secret name for connection string
Mode	—	—	✅	Spark read mode (default='PERMISSIVE')
Multiline	—	—	✅	Support multiline records (default=false)
NotebookTimeout	—	—	✅	Default timeout for notebooks (default=1800)
ProcessType	—	—	✅	Type of processing
Quote	—	—	✅	Quote character for CSV
QuoteMode	—	—	✅	Quote mode for CSV
SilverLoadOrder	—	—	✅	Load order for silver notebooks
SilverNotebookConcurrency	—	—	✅	Concurrency for silver notebooks (default=10)
SilverTableSettings	✅	✅	—	Table settings for Silver at connection level
SourceFolder	—	—	✅	Source folder path

---

Object/Table Properties

Defined in object YAML files (e.g., customers.yaml).

Property	API	Dataplatform	Wheel Package	Description
Connection	✅	✅	✅	Connection to use for this object
SourceTable	✅	✅	✅	Name of the object in the source
SourceSchema	✅	✅	—	Source schema (if applicable)
DataPlatformObjectName	✅	✅	✅	Override source name in dataplatform
Description	✅	✅	—	Description of the source
IsActive	✅	✅	✅	Set to active for generating (default=true)
KeepHistory	✅	✅	✅	History required in silver layer (default=true)
KeepOneDayRecords	—	—	✅	Keep only one day of records
Layers	✅	✅	✅	Which layers to generate (ALL/Bronze/Silver/Gold)
Prefix	✅	✅	—	Set a prefix for this object
Columns / Fields	✅	✅	✅	List of columns/fields from source
BronzeTableSettings	✅	✅	—	Table settings for Bronze at object level
SilverTableSettings	✅	✅	—	Table settings for Silver at object level
SourceFilter	—	—	✅	Filter query for source data
SourceFolder	—	—	✅	Override source folder
SourceOrder	—	—	✅	Order by clause for source
BronzeFolder	—	—	✅	Override bronze folder
BronzeKeyViolationAction	—	—	✅	Override connection setting
BronzeLoadOrder	—	—	✅	Override connection load order
BronzeLoadSkip	—	—	✅	Skip bronze load for this table
BronzeLoadViolationAction	—	—	✅	Action on load violations (continue/stop)
BronzeMaxExpectedDuration	—	—	✅	Max expected duration for bronze (default=1800)
DateFormat	—	—	✅	Override date format for this table
FileExtension	—	—	✅	Override file extension
FileType	—	—	✅	Override file type
HistorySettings	—	—	✅	History-specific settings
IgnoreNullKey	—	—	✅	Override ignore null key setting
PreBronzeNotebook	—	—	✅	Notebook to run before bronze
MidBronzeNotebook	—	—	✅	Notebook to run during bronze
PostBronzeNotebook	—	—	✅	Notebook to run after bronze
PreSilverNotebook	—	—	✅	Notebook to run before silver
MidSilverNotebook	—	—	✅	Notebook to run during silver
PostSilverNotebook	—	—	✅	Notebook to run after silver
BronzeNotebook	—	—	✅	Custom bronze notebook
SilverNotebook	—	—	✅	Custom silver notebook
SheetName	—	—	✅	Excel sheet name
SilverLoadOrder	—	—	✅	Override silver load order
SilverLoadType	—	—	✅	Load type (full/incremental, default='full')
SilverMaxExpectedDuration	—	—	✅	Max expected duration for silver (default=1800)
Trigger	—	—	✅	Trigger configuration
XmlRowPath	—	—	✅	XML row path for parsing

---

Column/Field Properties

Defined within the Columns or Fields array in object files.

Property	API	Dataplatform	Wheel Package	Description
SourceColumn	✅	✅	✅	Name of the attribute in source system
SourceDataType	✅	✅	✅	Datatype of the source
DataPlatformName	✅	✅	✅	Override name used in dataplatform
IsPrimaryKey	✅	✅	✅	Attribute is part of primary key (default=false)
IsNullable	✅	✅	✅	Attribute can be null (default=true for non-PK)
IsActive	✅	✅	✅	Attribute is active (default=true)
IsSilverOnly	✅	✅	✅	Attribute only generated for Silver (default=false)
IsPartition	✅	✅	—	Attribute is part of partition column (default=false)
SilverDataType	✅	✅	✅	Datatype used in Silver layer
SilverExpression	✅	✅	✅	Expression to fill Silver column
SilverConvertedColumn	✅	✅	✅	Create converted column (default=true)
SourceExpression	✅	✅	✅	Expression for Bronze layer conversion
Classification	✅	✅	—	Field classification (sensitive/restricted/internal/public)
HstgType	✅	✅	—	Type used in HSTG (historization)
DefaultValue	—	—	✅	Default value for column
DateFormat	—	—	✅	Date format for this column
DestinationColumn	—	—	✅	Destination column name (legacy)
ConversionExpression	—	—	✅	Conversion expression (legacy)
IsZOrder	—	—	✅	Use column for Z-ordering (default=false)
IsDeletedIdentifier	—	—	✅	Column identifies deleted records (default=false)

---

History Settings Properties

Nested within HistorySettings in table configuration.

Property	API	Dataplatform	Wheel Package	Description
BronzeDeleteFilterQuery	—	—	✅	Filter query for bronze delete operations
BronzeSkipDelete	—	—	✅	Skip delete operations in bronze (default=false)
SourceFilterQuery	—	—	✅	Filter query for source data
SilverFilterQuery	—	—	✅	Filter query for silver data

---

Notebook Configuration Properties

Nested within notebook configuration objects (e.g., PreBronzeNotebook).

Property	API	Dataplatform	Wheel Package	Description
Notebook	—	—	✅	Notebook name to execute
Timeout	—	—	✅	Timeout for notebook execution
Param001 - Param003	—	—	✅	Parameters to pass to notebook

---

Trigger Properties

Nested within Trigger configuration.

Property	API	Dataplatform	Wheel Package	Description
Notebook	—	—	✅	Notebook to trigger
NotifyWebhookUrl	—	—	✅	Webhook URL for notifications
NotifyWebhookType	—	—	✅	Type of webhook notification

---

Lakehouse Properties

Defined within the Lakehouses array in environment configuration.

Property	API	Dataplatform	Wheel Package	Description
Lakehouse	✅	✅	✅	Name of the lakehouse
Layer	✅	✅	✅	Layer (Bronze/Silver/Gold/Meta)
Workspace	✅	✅	✅	Fabric workspace name
UseSchema	✅	✅	—	Use schema for this lakehouse (default=true)
DefaultSchema	✅	✅	—	Schema to use (default='dbo' or 'his' for history)

---

Table Settings Properties

Nested within BronzeTableSettings or SilverTableSettings. These accept any valid Delta Lake table properties.

Property	API	Dataplatform	Wheel Package	Description
delta.enableChangeDataFeed	✅	✅	—	Enable change data feed
delta.autoOptimize.optimizeWrite	✅	✅	—	Enable optimize write
delta.autoOptimize.autoCompact	✅	✅	—	Enable auto compaction
delta.logRetentionDuration	✅	✅	—	Log retention duration
delta.deletedFileRetentionDuration	✅	✅	—	Deleted file retention
spark.sql.parquet.vorder.default	✅	✅	—	Enable V-Order
delta.stats.extended.collect	✅	✅	—	Collect extended statistics
delta.stats.extended.inject	✅	✅	—	Inject extended statistics
(Any Delta property)	✅	✅	—	FabricTableSettings accepts arbitrary Delta properties

---

Notes

1. API receives YAML converted to JSON in web requests and uses C# model classes to deserialize
2. Dataplatform reads YAML files directly from disk and uses the same C# model classes
3. Wheel Package reads YAML files at runtime in notebooks using Python dataclasses
4. Properties marked with ✅ are actively used by that component
5. Properties marked with — are not used by that component
6. Table Settings can be defined at three levels (Configuration → Connection → Object) with object-level overriding connection-level, which overrides configuration-level
7. The Dataplatform and API share the same model classes, so they support identical properties for object/field/configuration definitions
8. The Wheel Package has additional runtime properties for orchestration, notebook execution, and data loading that are not needed during generation