[BUG] Roo code reindex my large codebase everytime I open my repo

[b1231227]

Problem (one or two sentences)

Roo code reindex my large codebase everytime I open my repo.
The issue still occurs in ROO CODE 3.52.1.

Context (who is affected and when)

Restart VS Code and return to the previously indexed workspace.

Reproduction steps

Restart VS Code and return to the previously indexed workspace.

Expected result

The already indexed codebase should be reused without requiring a full reindex. Only modified files should trigger partial reindexing. The existing vector database should store and reload the index across sessions.

Actual result

Roo code reindex my large codebase everytime I open my repo.

Variations tried (optional)

No response

App Version

v3.52.1

API Provider (optional)

None

Model Used (optional)

N/A

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote-v0]

I've analyzed the codebase indexing system and wanted to share my findings and ask a few clarifying questions before proposing a fix.

Analysis

The code indexing system in Roo Code already has an incremental scan path (in src/services/code-index/orchestrator.ts). When VS Code restarts, it checks hasIndexedData() on the Qdrant vector store -- if existing indexed data is found, it runs an incremental scan that only re-embeds new/changed files (using SHA-256 content hashes stored in a persistent cache file).

However, I've identified several scenarios where this incremental path could fail, causing a full reindex:

1. Qdrant data not persisting -- If your Qdrant instance runs in-memory mode (the default for Qdrant Docker without a volume mount), all indexed data is lost when Qdrant restarts. This would cause hasIndexedData() to return false, triggering a full scan every time.

2. Qdrant not ready at VS Code startup -- If Qdrant is slow to start or temporarily unreachable when VS Code activates, hasIndexedData() catches the connection error and returns false, which triggers a full scan instead of waiting and retrying.

3. Aggressive error cleanup -- If any error occurs during indexing (even a transient API failure on a single file), the error handler in the orchestrator clears both the Qdrant collection AND the local cache file (clearCollection() + clearCacheFile()). This means any interrupted indexing session requires a complete restart from scratch.

4. Incremental scan still reads all files -- Even the incremental scan path reads every file in the workspace to compute SHA-256 hashes for comparison against the cache. For large codebases, this I/O-intensive process could appear slow and look like a "full reindex" in the UI, even though it only re-embeds changed files.

Questions

To narrow down the root cause:

• How is your Qdrant instance configured? Specifically, is it running via Docker, and does it use persistent storage (volume mount)? If Qdrant data is lost between sessions, that would directly cause this issue.
• Does the reindexing actually re-embed all files, or does it just show an "Indexing" status briefly while checking files? If it completes quickly (just checking hashes), the system is working as designed but the UX could be improved. If it takes a long time and actually calls the embedding API for all files, that points to data loss.
• Have you seen any error messages related to Qdrant connection failures or indexing errors? Errors during indexing cause complete data loss in the current implementation.
• Approximately how many files are in your workspace? This helps gauge whether the incremental check itself is the bottleneck.

Proposed Plan (pending answers)

Based on my analysis, I'd propose these improvements:

1. Add Qdrant connection retry logic -- Instead of immediately falling back to a full scan when Qdrant is unreachable, retry the connection with exponential backoff.
2. Preserve cache on partial failures -- Stop clearing the local cache file on transient errors so incremental scans can recover from where they left off.
3. Add mtime-based fast-skip -- Store file modification timestamps alongside content hashes in the cache, so the incremental scan can skip unchanged files without reading their full contents. This would dramatically speed up the check for large codebases.
4. Improve status messaging -- Distinguish between "Checking for changes" (incremental scan) and "Indexing" (actually embedding files) in the UI, so users know the system is reusing existing data.

Please mention @roomote in a comment with answers to the questions above (or approval of the plan if the root cause is already clear to you), and I'll proceed with the implementation.

[b1231227]

@roomote Running Qdrant via Docker, persistent storage exists. I see it creating a new Collections instance.

[roomote-v0]

Thanks for the details @b1231227! Since Qdrant has persistent storage but you're seeing a new collection created each time, I'll dig into why the existing collection isn't being reused. Will explore the collection naming/lookup logic and open a PR with a fix. Stay tuned.

[roomote-v0]

Opened a draft PR with my attempt at addressing this: #12147

The core issue was that getCollectionInfo() treated all Qdrant errors (including connection failures) as "collection not found", causing a new collection to be created even when one already existed. The error handler also aggressively wiped both the collection and cache on any indexing error, destroying valid pre-existing data.

The fix makes the code distinguish 404 (genuinely missing collection) from other errors, and only clears data when a brand-new collection fails -- existing collections are preserved. Feedback welcome!