2 min lesson
Why orchestration and why Dagster
Give the practical answer to "What is the core conceptual difference between Dagster's model and a task-centric orchestrator like Airflow?" Then name the detail that supports it.
Step 1 of 2
Orchestration is the layer that decides what runs, in what order, when and what to do when a step fails. On a lakehouse processing billions of events a day, it is the platform's nervous system.
Strip the buzzwords and an orchestrator does four things across the whole pipeline DAG: it sequences work so a transform never reads a half-written table, it schedules on a cadence or in response to events, it retries and recovers when a step dies and it observes every run so you know what is fresh, stale or broken. Without one you get a graveyard of cron jobs that nobody can reason about.
Dagster's distinctive bet is that you declare the data, not just the steps. In a task-centric tool like Airflow you write operators that run in an order you wired by hand; the framework has no idea those tasks produce a bronze_events table or that gold_daily_active depends on it. Dagster inverts this with software-defined assets: you declare the table or file each function produces and what it reads and the dependency graph is the data graph.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
The same lakehouse pipeline, declared two ways - step the rows to feel why asset-centric wins here.
Learn more
Full explanation
That fit with a lakehouse is why this matters for Cursor
That fit with a lakehouse is why this matters for Cursor. Your assets span Databricks Spark jobs, dbt models and ingestion connectors and an asset graph gives you one control plane with lineage across all of them. You can stand in front of gold_retention and trace it back through silver to the raw Kafka topic without reading five repos.
Learn more
Optional practice