Skip to content

feat(code-intelligence): add generic LSP and search code-intelligence skill#1

Merged
antonbabenko merged 5 commits into
masterfrom
feat/code-intelligence
May 16, 2026
Merged

feat(code-intelligence): add generic LSP and search code-intelligence skill#1
antonbabenko merged 5 commits into
masterfrom
feat/code-intelligence

Conversation

@antonbabenko
Copy link
Copy Markdown
Owner

@antonbabenko antonbabenko commented May 16, 2026

Description

Plugin(s) affected: new plugin: code-intelligence (layer-1, generic)

Type of change: New plugin

Summary:
Adds the generic, language-agnostic code-intelligence discipline as an inline
plugin. It owns the cross-language layer (LSP-vs-text-vs-fuzzy precedence,
position-anchored LSP calls, degradation gate, first-line tool-substitution
disclosure, anti-phantom-shim proof). Language-specific skills (terraform-skill)
extend it; this PR does not modify them. Generic only: no terraform-ls
specifics, no private fixtures, no private global-rule duplication, no
host-specific tool names.

Base note: this branch sits on the repo scaffold commit 2da4490 (marketplace
manifest, CI workflows, contributor docs) which master previously lacked.

Conventional Commit

Planned commit subject: feat(code-intelligence): add generic LSP and search code-intelligence skill
-> minor bump for code-intelligence post-merge (0.1.0 seed -> CI bumps + tags code-intelligence-v0.2.0).

Testing Evidence (REQUIRED for content changes)

Three baseline scenarios are defined in
plugins/code-intelligence/tests/baseline-scenarios.md
(find-callers+rename, tool-claimed-missing, fuzzy discovery). Baseline (WITHOUT)
and improved (WITH) transcripts are PENDING a manual run and will be attached
before merge.

  • Agent references new content
  • Agent applies new patterns proactively
  • No new rationalizations introduced

Standards Compliance Checklist

Frontmatter

  • name and description present; name letters/numbers/hyphens only
  • Description starts with "Use when..." and is < 1024 chars (251)
  • metadata.version 0.1.0 matches manifest plugin version (seed; CI owns thereafter)

Token Efficiency & Content Quality

  • SKILL.md lean (94 lines)
  • Detailed content in references/*.md (each subsection well under the ~1,600 char budget)
  • Tables over prose; imperative voice; DO vs DON'T
  • LLM Consumption Rules followed (decision-table-first, anchor stability, retrieval-first)

Marketplace / Structure (inline plugin)

  • plugins[] entry with source: ./plugins/code-intelligence
  • plugins/code-intelligence/skills/code-intelligence/SKILL.md exists
  • plugins/code-intelligence/CHANGELOG.md exists
  • Only the 0.1.0 seed set; no further version hand-edits (CI owns them)

Validation

  • Frontmatter validation passes
  • Manifest <-> SKILL.md version sync passes (0.1.0 == 0.1.0)
  • No broken internal links; every SKILL.md #anchor resolves to an existing ### heading
  • Generic-purity guard: no terraform/private-tool/private-rule leakage
  • Typography: ASCII clean (no em/en dash, curly quotes, ellipsis)
  • Markdown lint: advisory only (CI continue-on-error). MD013/MD024 are
    consistent with the repo's existing docs and the multi-scenario format;
    MD040 fixed.

For Maintainers

  • Testing evidence (baseline -> improved) to be attached before merge
  • Standards compliance verified
  • Squash with subject feat(code-intelligence): add generic LSP and search code-intelligence skill

… skill

New inline plugin: language-agnostic code-intelligence discipline.

- SKILL.md: tool-precedence decision table (LSP for symbol
  semantics, rg for exact text, semantic search for fuzzy
  discovery), position-anchored LSP calls, 3-point degradation
  gate, first-line substitution disclosure, anti-phantom-shim
  proof. Scope note: packaging is not enforcement.
- references/: tool-precedence.md, lsp-calls.md,
  degradation-and-disclosure.md (stable ### anchors, each
  well under the token budget).
- tests/baseline-scenarios.md: 3 generic scenarios
  (find-callers+rename, tool-claimed-missing, fuzzy discovery).
- CHANGELOG.md stub (CI-managed).
- marketplace.json: one appended local plugin entry, seeded
  at 0.1.0; root version and external terraform-skill entry
  untouched.

Generic only: no terraform-ls specifics, no CTF fixtures, no
private global-rule duplication, no host-specific tool names.
Language-specific skills extend this; a PreToolUse/subagent
enforcement gate remains a separate follow-up.
@antonbabenko
Copy link
Copy Markdown
Owner Author

Testing Evidence (baseline WITHOUT vs WITH skill)

Method: two independent general-purpose subagents, no repo context (methodology
described, not executed). Baseline agent had NO skill loaded. With-skill agent
was given the code-intelligence SKILL content and told to apply it. This is a
genuine A/B - the baseline agent never saw the discipline.

Scenario 1 - Find callers and rename safely

WITHOUT skill: Grep parseConfig -> Edit with replace_all: true per
file -> re-grep to confirm. Text-level replace; relies on judgement to skip
comments/strings; no semantic scoping.

WITH skill: text-search to anchor a position -> documentSymbol
(responsiveness) -> position-anchored LSP findReferences (retry once) -> Read
each location, exclude same-named-but-distinct symbols -> fresh-Read before each
per-file edit (offset shift) -> manual rename (no rename provider claimed).

Delta: blind text replace -> semantic reference set + offset-safe edits + false-positive exclusion. Success criteria met.

Scenario 2 - Tool claimed missing

WITHOUT skill: accepts "rg broken" at face value, switches to the Grep
tool, discloses the substitution but never verifies the claim.

WITH skill: disproves the claim first (type -a rg, ls -l resolved path,
rg --version banner) before any substitution; only a disclosed first-line
fallback if rg is genuinely absent; refuses a neural tool for "every call"
(drops exact matches). Standard disclosure format used.

Delta: unverified acceptance -> anti-phantom-shim proof before substitution. Success criteria met.

Scenario 3 - Fuzzy discovery

WITHOUT skill: broad Grep/Glob keyword sweep (auth, login, ...),
presents hits as the answer.

WITH skill: routes to the semantic/neural tier for the conceptual question;
first-line disclosure if no semantic tool is available; narrows with LSP/rg;
makes no exhaustive/exact-count claim from fuzzy results.

Delta: keyword sweep presented as complete -> tiered discovery + completeness honesty. Success criteria met.

  • Agent references new content
  • Agent applies new patterns proactively
  • No new rationalizations introduced

Raw transcripts: /tmp/code-intelligence-eval-baseline.md, /tmp/code-intelligence-eval-withskill.md (not committed, per plan - no repo artifacts).

…benefits

- .markdownlint.jsonc: repo-wide config (MD013 off, MD024
  siblings_only, MD060 off). Resolves the 29 markdownlint
  annotations on PR #1 and the pre-existing failures in the
  repo's own docs; the CI step is advisory (continue-on-error)
  but the annotations are now clean.
- references/: heading levels h3 -> h2 (fix MD001
  heading-increment under the h1 title). Anchor slugs unchanged,
  every SKILL.md link still resolves.
- SKILL.md: token compression - drop the intro that restated
  the precedence table, fold the Scope note into one intro
  line, remove the References index that duplicated the
  per-section Detail links. 94 -> 78 lines, no substance lost.
- README.md: add code-intelligence to the plugin table and a
  "Why these plugins" section explaining the benefits.
…ename, docs)

GPT review (critical + should-fix) applied; Gemini cheap nits applied.

- Degradation gate (CRITICAL): split "no LSP at all" (genuine
  unavailability -> first-line disclosure + text search, gate does
  not apply) from "LSP callable but position-anchored call returns
  empty" (run the 3-point gate). SKILL.md + degradation-and-disclosure.md.
- Rename/callers: prefer the server's own `rename`/`prepareRename`
  and call hierarchy when advertised (language semantics); manual
  findReferences+edits only when genuinely unsupported. Fixed the
  references-vs-callers conflation in the precedence table.
- README: symlink example used an external plugin path; now points
  to the inline code-intelligence plugin and notes external plugins
  install from their own repo.
- baseline-scenarios.md: stale "no markdownlint config" note
  corrected; scenario-3 success criterion made conditional on the
  host providing a semantic tool.
- Nits: cross-link Semantic Search Scope -> Position Anchoring;
  .gitignore += .DS_Store, *.swp.

Dismissed: Gemini's automated-release version-regex concern (works;
scaffold-scope, not this plugin) and the cosmetic CLAUDE.md quoting.
@antonbabenko
Copy link
Copy Markdown
Owner Author

antonbabenko commented May 16, 2026

Full review (/ask-both: GPT + Gemini, independent) + resolution

GPT: REQUEST CHANGES (1 critical, 5 should-fix). Gemini: APPROVE (0 critical, 2 should-fix, 3 nits). No terraform/private-tool/private-rule leakage found by either.

Applied (commit fb299ed)

Severity Finding Fix
Critical (GPT) Degradation gate unreachable when the host exposes no LSP at all - agent had no sanctioned path Split into "no LSP at all" (genuine unavailability -> first-line disclosure + text search, gate N/A) vs "LSP callable but empty" (run the 3-point gate). SKILL.md + degradation-and-disclosure.md
Should-fix (GPT) findReferences conflated with callers Precedence table now separates "every reference" (all usages) from "callers specifically" (call hierarchy if supported, else filtered findReferences)
Should-fix (GPT) Rename guidance avoided the rename provider Prefer server rename/prepareRename when advertised; manual enumerate+edit only when unsupported (SKILL + lsp-calls + table)
Should-fix (GPT) README symlink example used external plugin path Points to inline code-intelligence; notes external plugins install from their own repo
Should-fix (GPT) baseline note claimed "no markdownlint config" (PR adds one) Note corrected
Should-fix (GPT) Scenario-3 criterion required semantic search unconditionally Made conditional on the host providing a semantic tool, else disclosed fallback
Nit (Gemini) Cross-ref missing Semantic Search Scope -> Position Anchoring link added
Nit (Gemini) OS artifacts .gitignore += .DS_Store, *.swp

Dismissed (with reason)

  • Gemini - automated-release.yml version-regex fragility: works today (only version: in the top frontmatter lines); it is scaffold scope, not this plugin, and not blocking. Tracked as a separate scaffold-hardening concern if frontmatter grows.
  • Gemini - CLAUDE.md:64 unquoted source example: cosmetic, pre-existing scaffold text; not worth churn in this PR.

Also in this PR since the last review

  • markdownlint: .markdownlint.jsonc added (MD013 off, MD024 siblings_only, MD060 off) + ref headings h3->h2 (MD001). The 29 "Validate Skill Files" annotations are now clean (CI step was already advisory).
  • SKILL.md token-compressed (removed table-restating intro, duplicate References index, scope scaffolding).
  • README: code-intelligence row + "Why these plugins" benefits section.

Re-verified: all reference links/anchors resolve, ASCII-clean, generic-purity guard clean, marketplace<->SKILL version sync 0.1.0, markdownlint CI rules 0.

…protocol)

Make behavioral regression tests mandatory and self-prompting for every
inline plugin, so new features (by anyone, including agents) must
self-verify.

- validate.yml: new required step "Validate inline plugin tests".
  Every plugins/*/skills/*/SKILL.md must have a sibling
  tests/baseline-scenarios.md containing >=1 "## Scenario",
  a "## Running These Tests" protocol, and "### Success Criteria".
  Missing/incomplete -> CI fails. Inline only; external plugins
  unaffected.
- baseline-scenarios.md: added "## Running These Tests" - the
  WITHOUT -> WITH -> compare -> all-scenarios-gate protocol, plus
  the rule to add/update a scenario for any new behavior.
- CONTRIBUTING.md: Testing section rewritten to "required", with
  the mandatory file structure (canonical example = code-intelligence)
  and the per-PR procedure; CI bullet updated; inline "Adding a
  Plugin" step 5.
- CLAUDE.md: inline "Adding a Plugin" step 5; "Testing Changes"
  rewritten as a required, CI-enforced loop.
- PULL_REQUEST_TEMPLATE.md: required checkboxes to run
  baseline-scenarios.md and confirm all scenarios pass.

Unscoped (ci:) - process/infra, no plugin release.
…error)

- CONTRIBUTING.md:83 skeleton fence given a language (text) - was
  the only remaining MD040 markdownlint annotation on the globbed
  files.
- validate.yml: removed continue-on-error from the Lint Markdown
  step. Every check in "Validate Skill Files" (frontmatter, tests
  present, manifest, version sync, broken links, markdown lint) is
  now blocking.
- CONTRIBUTING.md: document that "Validate Skill Files" is a
  required status check on master - PRs cannot merge while red.

Branch protection requiring this check is applied to master out of
band. Unscoped (ci:) - no plugin release.
@antonbabenko antonbabenko merged commit 3d6f897 into master May 16, 2026
1 check passed
@antonbabenko antonbabenko deleted the feat/code-intelligence branch May 16, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant