dbt (Data Build Tool) cookbook

9 recipes · 60 steps · about 42 minutes of reading

Build a dbt staging model from a raw SQL script
One SQL query, pulling straight from the raw orders table. Three other reports are already asking to copy it.
BeginnerRead 4 minDoing: an hour6 steps
Give dbt a dev target
I ran a dbt model against what I thought was my sandbox. It rebuilt three tables our dashboard reads from.
BeginnerRead 4 minDoing: an hour7 steps
Catch a broken foreign key with dbt's relationships test
A join between orders and customers is dropping rows, and nobody noticed until finance asked why the numbers didn't add up.
Beginner to intermediateRead 4 minDoing: an hour to set up, plus a couple of days watching it before you flip it to error6 steps
Turn repeated SQL into a dbt macro
The same currency-conversion CASE statement is pasted into six models, and I just fixed a bug in one copy without touching the other five.
Beginner to intermediateRead 5 minDoing: an afternoon6 steps
Writing a custom dbt test
None of the four built-in tests catches a completed order with a negative total. Finance just found one.
IntermediateRead 4 minDoing: a couple of hours to write and wire it up, then a short wait on the data owner's fix before the build runs clean7 steps
Debugging a circular dependency in your dbt DAG
dbt run just failed: Found a cycle. No idea which two models are pointing at each other.
IntermediateRead 6 minDoing: about an hour6 steps
Finding what a column change would break with dbt's lineage graph
Someone wants to rename a column. What breaks downstream if they do?
IntermediateRead 5 minDoing: an hour or two, more if a lot of nodes need checking by hand6 steps
Turn an expensive nightly full-refresh model into an incremental dbt model
The nightly job that rebuilds the whole orders table now takes ninety minutes, and the warehouse bill has three people asking questions.
Intermediate to advancedRead 5 minDoing: half a day to build and verify, plus a few nights watching the runtime and the late-arrival check before fully trusting it8 steps
Block a broken dbt model from reaching production with a CI check
A broken model made it to production last week, and the first anyone heard about it was a Slack message from the CEO's dashboard.
Intermediate to advancedRead 5 minDoing: a day to wire up and prove out, plus watching it catch or pass the first several real pull requests before fully trusting it8 steps

Cookbook badge

Finish all 9 recipes to earn this.

In these roadmaps

Data Engineerstep 10 of 16

dbt (Data Build Tool) cookbook

Build a dbt staging model from a raw SQL script

Give dbt a dev target

Catch a broken foreign key with dbt's relationships test

Turn repeated SQL into a dbt macro

Writing a custom dbt test

Debugging a circular dependency in your dbt DAG

Finding what a column change would break with dbt's lineage graph

Turn an expensive nightly full-refresh model into an incremental dbt model

Block a broken dbt model from reaching production with a CI check

In these roadmaps

dbt (Data Build Tool) cookbook

Build a dbt staging model from a raw SQL script

Give dbt a dev target

Catch a broken foreign key with dbt's relationships test

Turn repeated SQL into a dbt macro

Writing a custom dbt test

Debugging a circular dependency in your dbt DAG

Finding what a column change would break with dbt's lineage graph

Turn an expensive nightly full-refresh model into an incremental dbt model

Block a broken dbt model from reaching production with a CI check

In these roadmaps