From 9f950ab50b892db4e31e94ebe1c44d5cedc4dcf4 Mon Sep 17 00:00:00 2001 From: umi008 Date: Thu, 9 Jul 2026 09:50:12 -0600 Subject: [PATCH] fix(explore): surface camel-infix field definers on service-layer codebases (#1196) Multi-word field-name queries (e.g. `profileInfo isTrialEligible quotaInfo billingMethod`) used to return 134 unrelated symbols and bury the defining controller and service files under a dense neighbor. Three compounding retrieval bugs were dead on JS/TS/Ruby/Python service-layer codebases (where the camel-infix definer is a method, not a class): 1. context builder 5b (CamelCase-boundary LIKE) used `name.indexOf(titleCased)` after `titleCased.toLowerCase()`, so a multi-hump token ("Profileinfo") could never indexOf into "getProfileInfoV2". The SQL layer matched case-insensitively; the JS boundary check silently re-tightened that and dropped the row. Now case-insensitive, with a guard that the boundary char itself is uppercase so we still require a real hump. 2. context builder 5b AND 5c restricted the LIKE batch to declaration kinds (class/interface/struct/trait/protocol/enum/type_alias), so on a method-centric codebase they contributed nothing. Both now issue a separate LIKE batch for callable kinds (function/method/ component) and merge before scoring, with a per-batch 200-row cap so a hot single-word term can't crowd classes out. 3. explore's named-symbol seeding was exact-name only. A field-name token (profileInfo) has no node of its own, so it contributed zero seeds: no +50 file score, no entryFiles protection, no named-file sort. Added findCamelInfixCallables: a LIKE-based fallback that accepts prefix or camel-hump boundary matches on method/function/ component kinds, case-insensitive. Fires only when the exact lookup yields no callable (cands.length === 0), so well-named queries are unaffected. Exposed CodeGraph.findNodesByNameSubstring publicly so this can be reached through the public API. --- CHANGELOG.md | 1 + __tests__/context-multi-word-fields.test.ts | 132 ++++++++++++++++++++ src/context/index.ts | 52 ++++++-- src/index.ts | 14 +++ src/mcp/tools.ts | 58 ++++++++- 5 files changed, 243 insertions(+), 14 deletions(-) create mode 100644 __tests__/context-multi-word-fields.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b0b58b73..3405ac025 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - The safety watchdog no longer kills a healthy index on severely degraded storage. It used to judge liveness purely by the event loop, so one long database write on a struggling disk looked identical to a hung process and could get a valid, in-progress index terminated. During `codegraph index`/`codegraph init` the watchdog now also checks whether the index files on disk are advancing before it acts: slow-but-progressing work is left alone (bounded by a hard cap), while a genuinely hung process is still killed exactly as fast as before. (#1231) - Incremental sync now picks up cross-file relationships that only become resolvable after an edit — for example, when a file gains an export that another, unchanged file was already importing or calling. Previously the reference in the unchanged file was never revisited, so callers, impact, and flow results silently omitted the new edge (while status reported a clean index) until a full re-index. References that can't be resolved yet are now remembered and automatically retried whenever a change introduces a symbol that could satisfy them — this also covers a class gaining a new method that other files already call. Thanks @loadcosmos for the report with a minimal reproduction. (#1240) - The reverse case is fixed too: when an edit removes or moves a symbol (or deletes its file), callers in unchanged files now re-resolve during the same sync — rebinding to the symbol's new home when it moved, or waiting to reconnect automatically when it comes back — instead of silently losing their relationship until a full re-index. (#1240) +- `codegraph_explore` now surfaces the files that assemble a queried business field on service-layer codebases (#1196). Three compounding retrieval bugs were fixed together: the context builder's CamelCase-boundary LIKE step was case-sensitive after lowercasing interior humps so any multi-hump query token silently dropped, both the CamelCase and the compound-term steps restricted to declaration kinds (class/interface/struct/...) and so contributed nothing on JS/TS/Ruby/Python codebases where camel-infix definers are methods, and explore's named-symbol seeding was exact-name only so a field-name token (`profileInfo`) — which has no node of its own — contributed zero seeds and weak FTS sub-token hits took the render budget. Multi-word field queries like `profileInfo isTrialEligible quotaInfo billingMethod` now rank the defining controller and service files at the top instead of burying them under a dense neighbor. ## [1.4.0] - 2026-07-10 diff --git a/__tests__/context-multi-word-fields.test.ts b/__tests__/context-multi-word-fields.test.ts new file mode 100644 index 000000000..f9acdd0cd --- /dev/null +++ b/__tests__/context-multi-word-fields.test.ts @@ -0,0 +1,132 @@ +/** + * Regression coverage for #1196 — multi-word field-name queries never + * surfaced the defining files because three retrieval mechanisms were + * dead on service-layer codebases (camel-infix definers are methods, + * not classes): + * + * 1. The context builder's 5b CamelCase-boundary LIKE step used + * case-sensitive `indexOf` after lowercasing interior humps, so a + * titleCased token ("Profileinfo") never matched "getProfileInfoV2" + * in JS, and restricted kinds to declaration-only classes. + * 2. The 5c compound-term step shared the same declaration-only kinds + * whitelist, so it contributed nothing on method-centric codebases. + * 3. explore's named-symbol seeding was exact-name only — a field-name + * token (`profileInfo`) has no node of its own, so the seed was empty + * and weak FTS sub-token hits took the render budget instead. + * + * The mini-repo below models a service-layer JS codebase where every + * camel-infix definer is a method, then asserts the previously-missing + * methods surface for the field-name query. + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as path from 'path'; +import * as os from 'os'; +import CodeGraph from '../src/index'; + +describe('multi-word field-name queries surface defining files (#1196)', () => { + let testDir: string; + let cg: CodeGraph; + + beforeEach(async () => { + testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1196-')); + + // Service-layer JS: every "field" is a key on a response object built + // inside a method whose name CONTAINS the field at a camel-hump + // boundary. There are no class-level containers to fall back on. + const controllerDir = path.join(testDir, 'controller'); + const serviceDir = path.join(testDir, 'service'); + fs.mkdirSync(controllerDir, { recursive: true }); + fs.mkdirSync(serviceDir, { recursive: true }); + + fs.writeFileSync( + path.join(controllerDir, 'profileController.js'), + `function getProfileInfo(req, res) { + res.json({ profileInfo: { name: req.user.name, isTrialEligible: true } }); +} +function getProfileInfoV2(req, res) { + res.json({ profileInfo: { name: req.user.name, isTrialEligible: false } }); +} +module.exports = { getProfileInfo, getProfileInfoV2 }; +` + ); + + fs.writeFileSync( + path.join(serviceDir, 'billing.js'), + `function _getCustomerBillingMethods(accountId) { + return { billingMethod: 'card', quotaInfo: { used: 0 } }; +} +module.exports = { _getCustomerBillingMethods }; +` + ); + + cg = CodeGraph.initSync(testDir, { + config: { include: ['**/*.js'], exclude: [] }, + }); + await cg.indexAll(); + }); + + afterEach(() => { + if (cg) cg.destroy(); + if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true }); + }); + + it('5b: case-insensitive CamelCase-boundary check recovers getProfileInfoV2 from a "profileInfo" token', async () => { + // The defining file MUST surface — before the fix, the case-sensitive + // `indexOf(titleCased)` after `titleCased.toLowerCase()` silently + // dropped the match. (#1196 bug 1) + const sg = await cg.findRelevantContext('profileInfo'); + const names = Array.from(sg.nodes.values()).map((n) => n.name); + const files = Array.from(sg.nodes.values()).map((n) => n.filePath); + expect(names).toContain('getProfileInfoV2'); + expect(files.some((f) => f.endsWith('profileController.js'))).toBe(true); + }); + + it('5b+5c: callable kinds (method/function) participate in CamelCase matching on a service-layer codebase', async () => { + // Before the fix, the kind whitelist was class/interface/struct/... only, + // so on a JS-only codebase 5b and 5c contributed zero results. (#1196 bug 2) + const sg = await cg.findRelevantContext('billingMethod'); + const names = Array.from(sg.nodes.values()).map((n) => n.name); + const files = Array.from(sg.nodes.values()).map((n) => n.filePath); + expect(names).toContain('_getCustomerBillingMethods'); + expect(files.some((f) => f.endsWith('billing.js'))).toBe(true); + }); + + it('multi-word field query recovers BOTH defining methods (#1196 bug 3: named-symbol seeding)', async () => { + // The original failure: a 4-token bag of business field names returned + // 134 unrelated symbols across 26 files and never surfaced the two + // defining methods. Assert the field-name tokens SEED the methods + // that assemble them. (#1196 bug 3) + const sg = await cg.findRelevantContext('profileInfo isTrialEligible quotaInfo billingMethod'); + const names = Array.from(sg.nodes.values()).map((n) => n.name); + expect(names).toContain('getProfileInfoV2'); + expect(names).toContain('_getCustomerBillingMethods'); + }); + + it('camel-infix boundary: a token at a true prefix or camel-hump boundary wins; a mid-word lowercase run does not', async () => { + // Add a name that contains "profileinfo" as a MID-WORD LOWERCASE RUN + // (no camel hump) and a name that contains it at a TRUE PREFIX boundary. + // The first should NOT be a camel-infix hit; the second SHOULD. + fs.writeFileSync( + path.join(testDir, 'boundaries.js'), + `function getProfileInfoByName() { return 1; } // camel-hump boundary: matches +function reprofileinfoBogus() { return 1; } // mid-word lowercase run: no boundary +module.exports = { getProfileInfoByName, reprofileinfoBogus }; +` + ); + await cg.sync(); + + // Use a query that goes through the camel-infix fallback (no exact-name + // node exists for "profileinfo") by adding the lowercase token after + // a name that does match exactly. + const sg = await cg.findRelevantContext('getProfileInfo profileinfo'); + const names = Array.from(sg.nodes.values()).map((n) => n.name); + + // True camel-hump boundary is reachable. + expect(names).toContain('getProfileInfoByName'); + // Mid-word lowercase run is NOT — `reprofileinfo` in `reprofileinfoBogus` + // has a lowercase char before it and a lowercase char at the boundary, + // so the findCamelInfixCallables boundary filter drops it. + expect(names).not.toContain('reprofileinfoBogus'); + }); +}); diff --git a/src/context/index.ts b/src/context/index.ts index 68123c284..69d306ba1 100644 --- a/src/context/index.ts +++ b/src/context/index.ts @@ -749,6 +749,12 @@ export class ContextBuilder { if (symbolsFromQuery.length > 0) { const camelDefinitionKinds: NodeKind[] = ['class', 'interface', 'struct', 'trait', 'protocol', 'enum', 'type_alias']; + // Callable kinds participate too: in service-layer codebases (Express-style, + // Express) the camel-infix definers of a queried field are METHODS + // (profileInfo → getProfileInfoV2), not classes. Fetched as a separate + // LIKE batch so hot single-word terms can't crowd classes out of the + // length-ordered 200-row batch. (#1196) + const camelCallableKinds: NodeKind[] = ['function', 'method', 'component']; const camelSearchedTerms = new Set(); const searchIdSet = new Set(searchResults.map(r => r.node.id)); // Track per-node term hits for multi-term boosting @@ -766,18 +772,33 @@ export class ContextBuilder { // have hundreds of substring matches. The LIKE scan cost is the same // regardless of LIMIT (SQLite scans all matches to sort), so we fetch // generously and let path-relevance scoring pick the best ones. - const likeResults = this.queries.findNodesByNameSubstring(titleCased, { - limit: 200, - kinds: camelDefinitionKinds, - excludePrefix: true, - }); + // Two batches (declaration kinds, then callable kinds) so a hot + // single-word term can't crowd classes out of the 200-row window. + const likeResults = [ + ...this.queries.findNodesByNameSubstring(titleCased, { + limit: 200, + kinds: camelDefinitionKinds, + excludePrefix: true, + }), + ...this.queries.findNodesByNameSubstring(titleCased, { + limit: 200, + kinds: camelCallableKinds, + excludePrefix: true, + }), + ]; // Filter to CamelCase boundaries, score by path relevance, and take top N const termCandidates: SearchResult[] = []; for (const r of likeResults) { const name = r.node.name; - const idx = name.indexOf(titleCased); + // Case-insensitive: title-casing lowercases interior humps + // ("profileInfo" → "Profileinfo"), which a case-sensitive indexOf + // can never find inside "getProfileInfoV2". SQL LIKE already matched + // case-insensitively; the JS boundary check must not silently + // re-tighten that. (#1196) + const idx = name.toLowerCase().indexOf(titleCased.toLowerCase()); if (idx <= 0) continue; + if (!/[A-Z]/.test(name.charAt(idx))) continue; // Accept CamelCase boundary (lowercase before match) OR // acronym boundary (uppercase before match, e.g., RPCProtocol) if (!/[a-zA-Z]/.test(name.charAt(idx - 1))) continue; @@ -841,11 +862,20 @@ export class ContextBuilder { const titleCased = sym.charAt(0).toUpperCase() + sym.slice(1).toLowerCase(); if (titleCased.length < 3) continue; - const likeResults = this.queries.findNodesByNameSubstring(titleCased, { - limit: 200, - kinds: camelDefinitionKinds, - excludePrefix: false, - }); + // Callable kinds participate here too: service-layer definers of a + // queried field are METHODS, not classes. (#1196) + const likeResults = [ + ...this.queries.findNodesByNameSubstring(titleCased, { + limit: 200, + kinds: camelDefinitionKinds, + excludePrefix: false, + }), + ...this.queries.findNodesByNameSubstring(titleCased, { + limit: 200, + kinds: camelCallableKinds, + excludePrefix: false, + }), + ]; for (const r of likeResults) { if (searchIdSet.has(r.node.id)) continue; diff --git a/src/index.ts b/src/index.ts index eacf3128a..a1e09690b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1101,6 +1101,20 @@ export class CodeGraph { return this.queries.getNodesByNamePrefix(prefix, limit); } + /** + * Substring search via LIKE (case-insensitive) — recovers camel-infix + * matches FTS can't (`Search` inside `TransportSearchAction`, `ProfileInfo` + * inside `getProfileInfoV2`). Ordered by name length so the shortest, most + * specific match wins. Exposed publicly so explore's named-symbol seeding + * can use it as a fallback for field-name tokens (#1196). + */ + findNodesByNameSubstring( + substring: string, + options: SearchOptions & { excludePrefix?: boolean } = {}, + ): SearchResult[] { + return this.queries.findNodesByNameSubstring(substring, options); + } + /** * Search nodes by text */ diff --git a/src/mcp/tools.ts b/src/mcp/tools.ts index d1629b2cb..fd72f059c 100644 --- a/src/mcp/tools.ts +++ b/src/mcp/tools.ts @@ -2619,9 +2619,21 @@ export class ToolHandler { // codegraph_node's findSymbolMatches.) Qualified tokens keep findAllSymbols. const isQual = /[.\/]|::/.test(t); const raw = isQual ? this.findAllSymbols(cg, t).nodes : cg.getNodesByName(t); - const cands = raw - .filter((n) => CALLABLE.has(n.kind) && !isTestPath(n.filePath)) - .sort((a, b) => (bodyLines(b) > 1 ? 1 : 0) - (bodyLines(a) > 1 ? 1 : 0) || bodyLines(b) - bodyLines(a)); + const toCands = (ns: Node[]) => + ns + .filter((n) => CALLABLE.has(n.kind) && !isTestPath(n.filePath)) + .sort((a, b) => (bodyLines(b) > 1 ? 1 : 0) - (bodyLines(a) > 1 ? 1 : 0) || bodyLines(b) - bodyLines(a)); + let cands = toCands(raw); + // Camel-infix fallback: a token that names a data FIELD or camel + // fragment (profileInfo, billingMethod) often has no node of its + // own — the definers are the callables whose NAME contains it at a + // camel/prefix boundary (getProfileInfoV2, _getCustomerBillingMethods). + // Exact-name lookup finds nothing, so without this the query's most + // load-bearing tokens contribute zero seeds and weak sub-token FTS + // hits take the render budget instead. (#1196) + if (!isQual && cands.length === 0 && t.length >= 4) { + cands = toCands(this.findCamelInfixCallables(cg, t)); + } // A specific name (<=3 defs) injects all its defs. An overloaded name // (`validate` = 10, `request` = 44) would flood the subgraph, so inject // only: the overloads whose file/class the query ALSO names (the agent @@ -4481,6 +4493,46 @@ export class ToolHandler { return { nodes: ranked.map(r => r.node), note }; } + /** + * Resolve a query token with NO node of its own (a data field / camel + * fragment like `profileInfo`) to the callables whose name contains it at + * a prefix or camel-hump boundary (`getProfileInfoV2`, + * `_getCustomerBillingMethods`), case-insensitively. Used by explore's + * named-symbol seeding so field-name tokens still seed the files that + * define them. FTS can't do this: camelCase identifiers are single FTS + * tokens, so `"profileinfo"*` never matches an interior hump. (#1196) + */ + private findCamelInfixCallables(cg: CodeGraph, token: string): Node[] { + // `constructor` is not a NodeKind — constructors are extracted as `method` + // nodes named `constructor`. The CAMEL_INFIX_KINDS list mirrors the + // service-layer kinds the issue calls out (Express-style method defs). + const CAMEL_INFIX_KINDS: NodeKind[] = ['method', 'function', 'component']; + let rows: Array<{ node: Node }> = []; + try { + rows = cg.findNodesByNameSubstring(token, { + limit: 50, + kinds: CAMEL_INFIX_KINDS, + }); + } catch { + return []; + } + const tl = token.toLowerCase(); + const out: Node[] = []; + for (const r of rows) { + const name = r.node.name; + const idx = name.toLowerCase().indexOf(tl); + if (idx < 0) continue; + if (idx > 0) { + // An interior match must start at a camel hump ("…ProfileInfo…") or + // right after a separator ("_profile_info"); a mid-word lowercase + // run ("reprofileinfo" for "profileinfo") is a false hit. (#1196) + if (/[a-zA-Z]/.test(name.charAt(idx - 1)) && !/[A-Z]/.test(name.charAt(idx))) continue; + } + out.push(r.node); + } + return out; + } + /** * Truncate output if it exceeds the maximum length */