Handbook contents

Learn 05~9 min

How models depend on each other

source() and ref() do more than replace hardcoded table names: they turn the project into a dependency graph that dbt can build, test and explain.

Give dbt the dependency

In a Snowflake worksheet you would write from STAGING.CSDS.STG_CSDS_BRIDGING. In dbt you write:

from {{ ref('stg_csds_bridging') }}

Use ref() when a query reads another model, including in joins and subqueries. A model reading two upstream models looks like this:

select
    wl.sk_patient_id,
    wl.week_ending_date,
    dict.specialty_name
from {{ ref('stg_wl_openpathways_data') }} wl
left join {{ ref('stg_dictionary_dbo_specialties') }} dict
    on wl.treatment_function_code = dict.bk_specialty_code
    and dict.is_treatment_function = true

The braces mark a template expression. dbt evaluates ref() before Snowflake runs the SQL. The quoted name identifies another model; it is not text to search for in the warehouse. Each call does two things:

  • dbt resolves the location for you. Developing, it points at the DEV__ databases; in production, the production ones. Same SQL, every environment.
  • dbt records the dependency. Your model now officially sits downstream of stg_wl_openpathways_data and the specialty lookup. It appears in lineage, runs in the right order, and anyone changing that staging model can see you depend on it.

How the resolution works

The mechanism has three steps. When dbt starts, it parses every file in the project, extracting the ref() and source() calls without running the model query. The declared references let it assemble the dependency graph. It then compiles each model, replacing every call with a fully qualified table name. Only then does it run anything, walking the graph in dependency order.

The name a call compiles to depends on the target. A target is a named set of connection settings in the project's profiles.yml: which Snowflake account and role dbt connects with, and which database settings it uses. Project naming rules also determine the final locations. The project uses a development target that points at the DEV__ databases; the scheduled workflows run the same project with a prod target that points at the production ones. The same line compiles differently depending on which target runs it:

one line, two targets
-- in the model
from {{ ref('stg_csds_bridging') }}

-- compiled with the development target
from DEV__STAGING.CSDS.STG_CSDS_BRIDGING

-- compiled with the prod target (deploys and scheduled builds)
from STAGING.CSDS.STG_CSDS_BRIDGING

Your SQL refers to model names; the target and project configuration determine their warehouse locations. Normal development writes to the DEV__ databases. Snowflake roles and grants determine what you are allowed to read or write, so a target name alone is not a security boundary. The production chapter explains how validation can deliberately read unchanged production parents while building changed models in DEV.

Sources are the entry point

The feeds landing in DATA_LAKE and DATA_LAKE__NCL are declared in YAML under models/sources/, then referenced with source():

from {{ source('csds', 'ActiveSubmission') }}

In this project, only generated raw models call source(). Everything you write uses ref(). That keeps a single, stable source entry point. A feed change can still require downstream changes, but readers do not each need a separate connection to the physical source table. The Find your source field guide explains the mapping pipeline that produces source declarations. The official dbt documentation explains the general behaviour of ref() and source(); this project's raw-layer restriction is a local convention on top of it.

You may find older staging models that call source() directly. The source field guide explains the current route to follow when working on those models.

The DAG

From every ref() and source() call, dbt assembles the whole project into a directed acyclic graph. This is a real slice of ours:

A parent is an input to another model. Upstream means following those inputs towards their origins; downstream means following consumers towards their outputs. A directed acyclic graph, or DAG, has arrows showing that direction and no circular dependencies.

DATA_LAKE.WLDATA_LAKE.DictionaryDATA_LAKE.CSDSraw_wl_openpathwaysraw_dictionary_specialtiesraw_csds_bridgingstg_wl_openpathways_datastg_dictionary_specialtiesstg_csds_bridgingint_wl_currentreporting layer →
hover a node to trace its lineage · dotted = source, solid = dbt model

The DAG is what makes dbt build -s +my_model possible: the +means "and everything upstream", and dbt knows exactly what that is. It is also why dbt rejects circular model dependencies: neither model can be built first if each requires the other.

The same graph is a discovery tool. The finding models lesson shows how to inspect ancestors and descendants before deciding that a new model is needed.

Dependencies and selection are separate

A reference tells dbt that a model needs another relation. It does not mean every command rebuilds that relation. dbt build -s my_model selects that model and the tests included by test selection. Its parents must already be available unless the command also selects them.

Think of two questions: what does this model depend on, and what does this run include? The graph answers the first; selectors answer the second. Building and checking a change returns to that distinction with a complete development example.

Quiz

0/4 answered
  1. 1Why is hardcoding STAGING.CSDS.STG_X a problem?

  2. 2Where are you allowed to use source()?

  3. 3ref('stg_csds_bridging') compiles to DEV__STAGING.CSDS.STG_CSDS_BRIDGING on your machine and STAGING.CSDS.STG_CSDS_BRIDGING in production. What decides?

  4. 4What does the + in dbt build -s +int_wl_current select?