From merge to production

Four workflows operate the project

The production lifecycle is defined in .github/workflows/. The workflow files are committed with the dbt project, so changes to the way models are checked or deployed go through the same Git review as model changes.

These workflows run dbt Fusion directly on GitHub-hosted runners and connect to Snowflake with the project's service credentials. Their production jobs share one dbt-prod concurrency queue, so a deploy and a scheduled build cannot write to production at the same time.

Development builds go to the DEV__ databases

The project's profiles.yml defines development, production and compile-only targets. When you run dbt build with the normal development target, the project naming macros route relations to mirror databases such as DEV__STAGING, DEV__MODELLING and DEV__REPORTING. The data lake is the one exception — it has no mirror, so development reads real source data in place while writing only to the DEV__ databases. Production workflows use the prod target and build the supported production relations.

This routing is what keeps normal local and PR builds away from the tables consumers use. Snowflake roles and grants remain the underlying permission boundary — targets choose destinations; grants decide permissions. In normal work, use the development target locally and let the workflows perform production builds.

Every PR commit passes the compile gate

dbt-compile.yml runs when a relevant pull request is opened, reopened or updated. It executes dbt compile --target ci-dev. This is read-only: it renders the project, reads Snowflake catalogue metadata and analyses the compiled SQL without materialising models.

Fusion includes a SQL engine rather than treating the model body as an opaque string. The compile gate can therefore reject invalid SQL as well as broken ref() and source() calls or invalid YAML and Jinja. Where the target catalogue contains the required metadata, it can also identify unresolved columns and incompatible types. When GitHub creates a merge-queue run, the same workflow compiles with ci-prod so the final queued change is analysed against production metadata before it reaches main.

What compile does not establish is how the query behaves when Snowflake executes it against real data. It does not materialise the relation, count its rows or run its data tests. The conditional validation workflow performs that Snowflake build and exercises the model's data contracts.

Snowflake PR validation is conditional

Unlike the compile gate, full Snowflake validation does not run automatically for every pull-request update. It runs only when review is requested, when the ❄️snowflake-ci label is added, or when somebody dispatches the workflow manually. A new trigger rebuilds the PR's complete changed set relative to main; a newer run for the same PR cancels the older one.

The workflow collects changed model SQL, the SQL models associated with changed YAML, seeds and snapshots, and models that use a changed macro. For an ordinary change it builds those selected nodes with the development target. If more than 100 nodes are selected, it switches to a full development build.

When a deployed manifest is available, validation adds --defer --state state --favor-state. Unselected parents then resolve to the production relations recorded in that manifest instead of relying on a complete or fresh DEV mirror. The changed nodes still write only to the DEV__ databases.

A green validation means the selected nodes built and their tests passed with the inputs used by that run. Review still has to establish that the definition, grain and change are the right ones. The Git and pull requests lesson explains how those forms of evidence come together before merge.

A merge deploys the changed graph

A push to main that changes models, macros, seeds, snapshots, packages or project configuration starts dbt-deploy.yml. The workflow fetches the state manifest published by the previous deploy and runs:

dbt build --target prod --select state:modified+ --state state

state:modified selects nodes whose compiled definition has changed. The trailing + also selects every descendant. A change to int_wl_current, for example, deploys that model and the waiting-list facts, provider summaries and published products that depend on it. If no state manifest exists, or the workflow is started manually, it performs a full production build.

The project publishes the deploy's manifest and run results after every non-cancelled attempt, including a failed attempt. This is a deliberate project behaviour: a failed node is attributed to the deploy that introduced it and is not retried by a later unrelated merge. A fix changes the node again, making it state:modified for the fixing deploy. Scheduled runs can retry transient failures on lineages they cover.

Merge commits can contain [skip deploy] or [skip-deploy], but that is an exceptional control rather than the normal route. Skipping means the code on main and the last deployed state intentionally differ; use it only when the change should genuinely not build models.

Five schedules keep production current

Deployment applies changed code. dbt-scheduled.yml then rebuilds the relevant parts of that code as new source data arrives. The current schedules are expressed in UTC:

A model tagged daily is selected by the daily build; the leading + also brings in the ancestors it needs. The weekly selection rebuilds from staging upwards, and the first-of-month run replaces that week's normal run with a full refresh. Scheduled runs do not publish the deployment state manifest: that manifest records what was deployed from main, not which relations happened to refresh most recently.

Failures become GitHub issues

Deploy and scheduled workflows pass their run results to report_dbt_failures.py. A failed scheduled run creates or updates one GitHub issue for that run type and names the failed nodes. Repeated failures add comments to the same issue; a later successful run closes it. The GitHub–Teams subscription carries issues labelled dbt-run-failure into the team channel. SDL intraday is currently excluded from this issue flow.

dbt skips descendants of a failed model or test, so those downstream relations normally retain their previous versions. The failed relation itself still needs inspection. In particular, a model can finish building and then fail a data test, leaving its new relation in place while its dbt descendants are skipped. A red run contains the failure; it does not roll the entire graph back.

The issue identifies what failed and links to the Actions run. Start with the first failed node rather than the skipped descendants, read its log and run result, reproduce the problem in development where possible, and send the correction through the normal branch and pull-request process. Deploy failures remain attached to the change that caused them until the tracked nodes build successfully again or leave the project.

What a model author is responsible for

The workflows remove manual deployment and scheduling, but the repository still needs enough information to operate a model correctly. Before merge, make sure the model:

• uses ref() and source() so its place in the deploy graph is visible;
• has tests that protect its grain and important business rules;
• has the correct tags for the cadence its consumers require;
• builds in development and has passed the appropriate PR validation;
• documents any unusual operational dependency or recovery step.

After merge, check the deploy when the change is time-sensitive or high-impact. If the model later appears in a failure issue, its SQL, description, tests, lineage and run artifacts should give the responding analyst enough context to act without reconstructing the original piece of work.

Deployment answers whether the change built. Ongoing health is a separate responsibility: Observing production explains how the Elementary-backed app connects later runs, failures, performance and downstream impact.