Skip to content

fix: nested non-DAG with-statement prematurely exits DAG context in version inflation checker#68795

Open
kalluripradeep wants to merge 2 commits into
apache:mainfrom
kalluripradeep:fix/dag-version-checker-nested-with-exits-dag-context
Open

fix: nested non-DAG with-statement prematurely exits DAG context in version inflation checker#68795
kalluripradeep wants to merge 2 commits into
apache:mainfrom
kalluripradeep:fix/dag-version-checker-nested-with-exits-dag-context

Conversation

@kalluripradeep

@kalluripradeep kalluripradeep commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Closes #68796

What's the problem?

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context() unconditionally at the end of every with-statement. When a non-DAG with-statement (e.g. with open(...) as f) was nested inside a DAG with-block, it would reset is_in_dag_context=False. Any tasks constructed after the nested with — but still inside the DAG block — were invisible to the checker and never flagged for runtime-varying values even when they used datetime.now(), random(), etc.

with DAG('my_dag') as dag:
    with open('file.txt') as f:   # <-- this incorrectly exited DAG context
        data = f.read()
    # BUG: datetime.now() not flagged — checker thinks we're outside DAG
    t1 = BashOperator(task_id='t', bash_command=str(datetime.now()))

Root cause

# visit_With (before fix) — unconditional, wrong
self.dag_detector.exit_dag_context()

Fix

Guard the call with if is_with_dag_context so only the with-statement that actually entered the DAG context exits it.

if is_with_dag_context:
    self.dag_detector.exit_dag_context()

Changes

  • airflow-core/src/airflow/utils/dag_version_inflation_checker.py — one-line guard
  • airflow-core/tests/unit/utils/test_dag_version_inflation_checker.py — 4 regression tests

…ersion inflation checker

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AirflowRuntimeVaryingValueChecker misses runtime-varying values in tasks after nested non-DAG with-statements

1 participant