Software-defined assets & dependencies

An asset is a persisted data object plus the function that produces it and a declaration of what it reads. Once you think in assets, the pipeline stops being a pile of scripts and becomes a graph you can query.

The mechanics are small. You decorate a function with @asset, name its upstream dependencies as parameters and return the data. Dagster records that this function produces, say, silver_events and consumes bronze_events. Materializing an asset means running its function and persisting the result.

from dagster import asset

@asset
def bronze_events(context) -> None:
    # land raw product telemetry as-is into Delta (schema-on-read)
    ...

@asset(deps=[bronze_events])
def silver_events(context) -> None:
    # clean, dedupe, enforce schema; one row per real event
    ...

@asset(deps=[silver_events])
def gold_daily_active(context) -> None:
    # business aggregate read by BI and reverse-ETL
    ...

Because the edges are declared, the asset graph hands you three things you would otherwise build by hand: end-to-end lineage, selective materialization (rerun silver_events and everything downstream, nothing else) and a computed answer to “what is stale?” when an upstream changes.