If using dbt indirect selection other than `eager` downstream assets are skipped but job still completes with "success"

[treilly1]

What's the issue?

Somewhere between dagster 1.11.5 and 1.12.8 a regression occurred which causes assets downstream of dbt models to skip. There is little warning since the job / runs still succeeds despite sometimes skipping large amounts of downstream assets. The issue seems rooted in dagster expecting all tests asset checks from eager selection vs DbtCliResource that may be pointed at a dbt_project.yml with indirect selection set to something other than eager.

Redacting because company stuff, but in dagster 1.11.5 downstream assets are not skipped if one of the dbt asset checks are skipped (intentionally via cautious indirect selection)

Whereas in dagster 1.12.8 (I also tested 1.12.12 and saw same thing) the same message about not yielding expected outputs appears, but it then skips downstream assets.

the downstream asset's event logs show

What did you expect to happen?

Same behavior as before. The downstream assets should not just skip when some tests don't run (especially when eager not used), or at least allow whether or not the asset checks are blocking to be configured somehow. If this change was intended, it should be noted in the changelog somewhere with tips on how to disable dbt asset checks if indirect selection other than eager is needed. See additional info section below for workaround.

How to reproduce?

configure you dbt_project.yml to set indirect selection to something other than eager (I tried both cautious and buildable and bug occurred in both)

flags:
  indirect_selection: cautious

Then start a run that has at least 2 assets:

1. a dbt asset (model_1) that has a test referencing another dbt asset (model_2) not part of the run selection
2. An asset downstream of model_1 that is NOT a dbt asset.

On dagster 1.11.5 you'll see in event log model_1 output a message like "op did not yield or return expected output" that will reference the test that depends on model_2, but the downstream non dbt asset will still run.

Do the same on dagster 1.12.8 you'll see the same "op did not yield or return expected output" message, but the downstream asset will skip. The downstream asset event log will show "Skipping step ... due to skipped dependencies" but the run still finishes with "Success".

Dagster version

1.12.8 but observed same behavior on 1.12.12

Deployment type

Local

Deployment details

Local docker compose setup, but observed same issues in deployed ECS environment.

Additional information

Was able to workaround this by defining

dagster_dbt_translator = DagsterDbtTranslator(
    settings=DagsterDbtTranslatorSettings(enable_asset_checks=False)
)

then passing that into each @dbt_assets like

@dbt_assets(
    manifest=dbt_manifest,
    dagster_dbt_translator=dagster_dbt_translator,
    ...
)

but would be nice to be able to load dbt tests as asset checks like before.

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.