Skip to content

Commit 6322dc0

Browse files
authored
perf(telemetry): unblock per-tool quality analysis (GDPR annotations + diagnostic fields + InvocationGuard) (#1650)
* feat(telemetry): add diagnostic fields, GDPR annotations, and InvocationGuard infra Telemetry contract changes to unblock per-tool quality analysis (driven by the 9-query deep-dive in 1. GDPR annotation blocks (the root cause of PR #1644 fields being filtered) - Added /* __GDPR__ ... */ blocks above the three sanitizedSend call sites (recordToolInvocation, recordChatActivation, recordLaunchInternal) so the ddtelfiltered cluster keeps new Properties keys instead of stripping them. 2. Five new enums to replace stringly-typed fields: - OperatingSystem: win | mac | linux | other - ClassNameDetectionStrategy: mavenStandard | gradleStandard | vscodeSrc | workspaceRoot - ClassNameDetectionFailure: sourceDirMissing | fileNotFound | parseError | noPackageDeclaration - SentinelOutcome: recorded | silentReturn | cancelled | exception 3. ToolInvocationRecord extended with platform/version context: - os, javaMajorVersion, projectSystem, retryCount, previousOutcome These split the per-tool funnel by OS / Java major / project system so we can confirm the Linux 4.3 percent vs Windows 27.7 percent start-rate divergence. 4. LaunchInternalEvent union extended with four new sentinel variants and elapsedMs/thresholdMs: - debugSession.sentinel: emitted on EVERY invoke so even infinite hangs leave a trail - debugSession.silentReturn: invoke returned with no outcome event (the 32.8 percent gap) - debugSession.cancelled: user cancelled mid-flight - debugSession.exception: handler threw an exception, classified - classNameDetection.failed: replaces collapsed boolean with strategy + failureReason - debugSessionStarted/Timeout now carry elapsedMs + thresholdMs for histograms 5. InvocationGuard infrastructure (beginDebugSessionInvocation()): - Returns markOutcomeRecorded / markException / close methods - Emits sentinel at begin; close auto-emits silentReturn if no outcome recorded - Lets us measure the 32.8 percent silent-loss bucket with closed-loop attribution 6. SessionInvocationTracker (nextAttempt / completeAttempt): - Per-window, per-tool in-memory retry counter - Stores ONLY enum (previousOutcome) and integer (count) — no user data - Quantifies the LM auto-retry-pays-off pattern (17.7 percent at 1 -> 64.9 percent at 6-10) Compiles clean (tsc --noEmit) and passes tslint. * feat(lmtool): wire diagnostic context, retry attribution, and bump timeouts Consumes the new telemetry contracts from the previous commit and threads them through all 10 LMT invoke handlers. Also tunes the two session-wait thresholds to cover the Started P95 latency observed in 30d telemetry. Timeout tuning (driven by 30d Started latency: P50=12s / P90=67s / P95=100s): SESSION_WAIT_TIMEOUT: 45000 -> 120000 (eventBased path, waitForSession=true) SMART_POLLING_MAX_WAIT: 15000 -> 90000 (default polling path) Previous values clipped ~10 percent of legitimate slow starts as timeouts and biased the two strategies against each other. Aligning at 120s/90s also lets us cleanly compare cross-strategy success rates. Per-tool context fields (added to all 10 recordToolInvocation call sites): os / javaMajorVersion: from getOs() / getJavaMajorVersion() getJavaMajorVersion probes the redhat.java extension API (cached), falls back to JAVA_VERSION env var, then 'unknown'. classifyJavaMajorVersion handles legacy '1.8.0_xxx' -> '8' and modern '21.0.1' -> '21'. retryCount / previousOutcome: from nextAttempt() / completeAttempt() Per-window, per-tool counter. Used to test the LM auto-retry-pays-off pattern. debug_java_application handler: Wraps the entire invoke in a beginDebugSessionInvocation() guard so even infinite hangs leave a sentinel + silentReturn trail. This is the closed-loop half of the 32.8 percent silent-loss bucket detected by D2 in the deep-dive. classNameDetection rewrite: findFullyQualifiedClassName() now returns { className, failureReason, strategy } instead of string|null. The failure branch emits the new classNameDetection.failed event with structured strategy + failureReason so we can distinguish sourceDirMissing / fileNotFound / parseError / noPackageDeclaration — the previous boolean detected:false collapsed all four causes. Timeout / Started events: debugSessionStarted.eventBased + debugSessionTimeout.eventBased + debugSessionTimeout.smartPolling now carry elapsedMs + thresholdMs so we can histogram the Started/Timeout latency distribution and verify the threshold bump moves the right mass. Compiles clean (tsc --noEmit) and passes tslint. * review: fix 4 reviewer-flagged issues Address review feedback on PR #1650: 1. InvocationGuard was effectively disabled — guard.markOutcomeRecorded() was called unconditionally in finally, so guard.close() could never emit debugSession.silentReturn / cancelled / exception. Moved the mark calls inline next to the four session-terminal recordLaunchInternal sites (debugSessionStarted.eventBased, debugSessionTimeout.eventBased, debugSessionDetected, debugSessionTimeout.smartPolling). Threaded guard parameter into debugJavaApplication. The catch path now lets close() emit debugSession.exception via markException as designed. 2. targetInfo formatting at the call site stringified the new structured ClassNameDetectionResult as [object Object]. Renamed local to 'detection' and explicitly read detection.className. The if-truthy branch was also always taken, suppressing the no-package warning. 3. The 'found file but no package' branch returned { className: simpleClassName } which made callers treat it as a successful detection, hiding the failure event. Now returns { className: null, failureReason: 'noPackageDeclaration' } so classNameDetection.failed fires. Behavior is preserved because the call site already falls back to input.target on null. 4. ClassNameDetectionResult is now a discriminated union (success has no failureReason; failure requires it). Eliminates the 'unused on success' sentinel and lets TS narrow at use sites. Required strict-null narrowing (className !== null) at the two call sites. tsc clean; tslint clean.
1 parent 809056d commit 6322dc0

2 files changed

Lines changed: 627 additions & 34 deletions

File tree

0 commit comments

Comments
 (0)