setting `exclude_gitignore = true` breaks stubtest

[jorenham]

When I add exclude_gitignore = true to the [tool.mypy] in the pyproject.toml of numtype, stubtest suddenly fails to find the numpy-stubs, while mypy itself still seems to works. This also occurs with an empty .gitignore.

Note that stubtest runs in an isolated uv environment (tool/stubtest.py), because without it, it doesn't work at all 🤷🏻.

See numpy/numtype#584 for the repro and full stubtest output.

[jorenham]

I'm able to repro this on scipy-stubs with the following command:

stubtest --allowlist=.mypyignore --mypy-config-file=pyproject.toml scipy

Error output (with --concise):

scipy._lib.decorator failed to find stubs
scipy._lib.deprecation failed to find stubs
scipy._lib.doccer failed to find stubs
scipy._lib.messagestream failed to find stubs
scipy._lib.uarray failed to find stubs
scipy.integrate._ivp.dop853_coefficients failed to find stubs
scipy.interpolate.dfitpack failed to find stubs
scipy.linalg.interpolative failed to find stubs
scipy.misc failed to find stubs
scipy.misc.common failed to find stubs
scipy.misc.doccer failed to find stubs
scipy.optimize._lsq.bvls failed to find stubs
scipy.optimize._lsq.common failed to find stubs
scipy.optimize._lsq.dogbox failed to find stubs
scipy.optimize._lsq.givens_elimination failed to find stubs
scipy.optimize._lsq.trf failed to find stubs
scipy.optimize._lsq.trf_linear failed to find stubs
scipy.optimize._trustregion_constr.canonical_constraint failed to find stubs
scipy.optimize._trustregion_constr.equality_constrained_sqp failed to find stubs
scipy.optimize._trustregion_constr.minimize_trustregion_constr failed to find stubs
scipy.optimize._trustregion_constr.projections failed to find stubs
scipy.optimize._trustregion_constr.qp_subproblem failed to find stubs
scipy.optimize._trustregion_constr.report failed to find stubs
scipy.optimize._trustregion_constr.tr_interior_point failed to find stubs
scipy.optimize.cython_optimize failed to find stubs
scipy.optimize.elementwise failed to find stubs
scipy.sparse.linalg._isolve.utils failed to find stubs
scipy.sparse.spfuncs failed to find stubs
scipy.special.add_newdocs failed to find stubs
scipy.special.basic failed to find stubs
scipy.special.cython_special failed to find stubs
scipy.special.orthogonal failed to find stubs
scipy.special.sf_error failed to find stubs
scipy.special.specfun failed to find stubs
scipy.special.spfun_stats failed to find stubs
scipy.stats._levy_stable.levyst failed to find stubs
scipy.stats._rcont.rcont failed to find stubs
scipy.stats._unuran.unuran_wrapper failed to find stubs
scipy.stats.sampling failed to find stubs
note: unused allowlist entry scipy(\.\w+)?\._typing
note: unused allowlist entry scipy\.linalg\._decomp_interpolative\.__all__
note: unused allowlist entry scipy\._lib\.decorator\.(DEF|ArgSpec|FunctionMaker|__init__|append|dispatch_on|get_init|getargspec|init|n_args)
note: unused allowlist entry scipy\.stats\._qmvnt\.phi(nv)?

This strange thing is that there are only 43 errors reported, even though there are more than 500 stubbed modules in scipy-stubs/.

Without exclude_gitignore = true in pyproject.toml, with exclude_gitignore = false, or without passing --mypy-config-file, this reports no errors.

[brianschubert]

Hmm, I'm having trouble reproducing this. Here's what I tried:

git clone https://github.com/scipy/scipy-stubs/
cd scipy-stubs
uv venv
uv pip install -e . --group type
uv pip install -e <path to mypy clone>

# Manually set `exclude_gitignore = true`
$ git diff
diff --git a/pyproject.toml b/pyproject.toml
index f9764bb..3e6555b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -97,6 +97,7 @@ strict = true
 disallow_subclassing_any = false
 enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"]
 warn_unreachable = false # required by repo-review
+exclude_gitignore = true
 
 [tool.pyright]
 exclude = [

Then running the stubtest command you provided seems to work (no failed to find stubs errors, just PEP 800 issues):

$ uv run --no-project stubtest --allowlist=.mypyignore --mypy-config-file=pyproject.toml --concise scipy
scipy._cyutility.Enum is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy._cyutility.array is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy._cyutility.memoryview is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy._lib._util.Generator is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.cluster._hierarchy.Heap is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.cluster._hierarchy.LinkageUnionFind is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.io.matlab._mio5_utils.VarHeader5 is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.io.matlab._mio5_utils.VarReader5 is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.io.matlab._streams.GenericStream is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.io.matlab._streams.ZlibInputStream is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.optimize._bglu_dense.BGLU is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.optimize._bglu_dense.LU is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.cKDTree is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial._ckdtree.cKDTree is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial._ckdtree.cKDTreeNode is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.ckdtree.cKDTree is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.kdtree.cKDTree is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform.RigidTransform is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform.Rotation is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform._rigid_transform.RigidTransform is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform._rigid_transform.Rotation is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform._rotation.Rotation is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.spatial.transform.rotation.Rotation is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._biasedurn._PyFishersNCHypergeometric is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._biasedurn._PyStochasticLib3 is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._biasedurn._PyStochasticLib3.__init__ is inconsistent, runtime does not have *args parameter "args"
scipy.stats._biasedurn._PyStochasticLib3.__init__ is inconsistent, runtime does not have **kwargs parameter "kwargs"
scipy.stats._biasedurn._PyWalleniusNCHypergeometric is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._levy_stable.levyst.Nolan is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._unuran.unuran_wrapper.DiscreteAliasUrn is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._unuran.unuran_wrapper.DiscreteGuideTable is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._unuran.unuran_wrapper.Method is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._unuran.unuran_wrapper.NumericalInverseHermite is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats._unuran.unuran_wrapper.TransformedDensityRejection is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats.sampling.DiscreteAliasUrn is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats.sampling.DiscreteGuideTable is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats.sampling.NumericalInverseHermite is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy.stats.sampling.TransformedDensityRejection is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub

I wonder if there might be a .gitignore file in a parent directory that's inferring in an unexpected way?

Would you be able to try running stubtest with this patch applied?

diff --git a/mypy/modulefinder.py b/mypy/modulefinder.py
index d61c9ee3e..bea92795d 100644
--- a/mypy/modulefinder.py
+++ b/mypy/modulefinder.py
@@ -715,7 +715,7 @@ def matches_gitignore(subpath: str, fscache: FileSystemCache, verbose: bool) ->
         if fscache.isdir(relative_path):
             relative_path = relative_path + "/"
         if gi_spec.match_file(relative_path):
-            if verbose:
+            if verbose or True:
                 print(
                     f"TRACE: Excluding {relative_path} (matches .gitignore) in {gi_path}",
                     file=sys.stderr,

[jorenham]

Hmm yea, your method also doesn't produce any errors for me. When I install scipy-stubs as editable, I don't see the problem. I only see it when I run it like

uv run --no-editable stubtest --allowlist=.mypyignore --mypy-config-file=pyproject.toml scipy

(Usually, for development purposes, I also add a --reinstall-package=scipy-stubs in there, but that shouldn't matter much here.)

The errors disappear when I run

uv run stubtest --allowlist=.mypyignore --mypy-config-file=pyproject.toml scipy

[jorenham]

Would you be able to try running stubtest with this patch applied?

diff --git a/mypy/modulefinder.py b/mypy/modulefinder.py
index d61c9ee3e..bea92795d 100644
--- a/mypy/modulefinder.py
+++ b/mypy/modulefinder.py
@@ -715,7 +715,7 @@ def matches_gitignore(subpath: str, fscache: FileSystemCache, verbose: bool) ->
if fscache.isdir(relative_path):
relative_path = relative_path + "/"
if gi_spec.match_file(relative_path):

•        if verbose:
  

•        if verbose or True:
             print(
                 f"TRACE: Excluding {relative_path} (matches .gitignore) in {gi_path}",
                 file=sys.stderr,
  

Hmm, that doesn't seem to make any difference for me

[jorenham]

I wonder if there might be a .gitignore file in a parent directory that's inferring in an unexpected way?

doesn't seem to be the case:

$ git status --ignored
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   pyproject.toml

Ignored files:
  (use "git add -f <file>..." to include in what will be committed)
        .mypy_cache/
        .venv/

no changes added to commit (use "git add" and/or "git commit -a")

[brianschubert]

Ignored files:
  (use "git add -f <file>..." to include in what will be committed)
        .mypy_cache/
        .venv/

Ah, is .venv/ 1) listed in .gitignore and 2) where the stub files are installed? I wonder if mypy's ignoring the stub files since they're in a venv directory that's excluded by .gitignore.

[brianschubert]

I can reproduce this by installing scipy-stubs into a venv at .venv/ (which is excluded by scipy-stub's .gitignore):

$ uv run --no-project stubtest --allowlist=.mypyignore --mypy-config-file=pyproject.toml --concise scipy
TRACE: Excluding .venv/lib/python3.13/site-packages/scipy-stubs/__config__.pyi (matches .gitignore) in /home/brian/Projects/open-contrib/mypy/temp/scipy-stubs
TRACE: Excluding .venv/lib/python3.13/site-packages/scipy-stubs/__init__.pyi (matches .gitignore) in /home/brian/Projects/open-contrib/mypy/temp/scipy-stubs
<snip>
TRACE: Excluding .venv/lib/python3.13/site-packages/scipy-stubs/stats/ (matches .gitignore) in /home/brian/Projects/open-contrib/mypy/temp/scipy-stubs
TRACE: Excluding .venv/lib/python3.13/site-packages/scipy-stubs/version.pyi (matches .gitignore) in /home/brian/Projects/open-contrib/mypy/temp/scipy-stubs
scipy._lib._util.Generator is a disjoint base at runtime, but isn't marked with @disjoint_base in the stub
scipy._lib.decorator failed to find stubs
scipy._lib.deprecation failed to find stubs
<snip>
scipy.stats._unuran.unuran_wrapper failed to find stubs
scipy.stats.sampling failed to find stubs

[cjwatson]

I wonder if #20775 might be relevant here? git status --ignored would be misleading in that case, because it (correctly) stops at the top level of the working tree while mypy currently doesn't.