feat(conversation_manager): add token-aware context management

[srbhsrkr]

Summary

Closes #2146
Addresses #1294, #555, #298
Related to #1295, #1678, #1296, #2048

• Add _token_utils.py with estimate_tokens (chars/4 heuristic) covering all ContentBlock types and TokenCounter type alias for pluggable token counting ([CONTEXT] [FEATURE] Token Estimation API #1294)
• Add max_context_tokens, token_counter, and compactable_after_messages to SlidingWindowConversationManager for token-budget enforcement and micro-compaction of stale tool results ([FEATURE] Proactive Context Compression #555, [FEATURE] In-envent-loop cycle context management #298)
• Add max_context_tokens, proactive_threshold, and token_counter to SummarizingConversationManager for proactive summarization when token threshold is exceeded ([FEATURE] Proactive Context Compression #555)
• Always use heuristic estimator (never stale model-reported latest_context_size) to prevent over-reduction spirals
• Hook calls apply_management() (not reduce_context() directly) to ensure micro-compaction runs before trimming

How this relates to existing issues

Issue	Status
#1294 — Token Estimation API	Addressed: estimate_tokens() + TokenCounter type on conversation managers. Complementary to a future Model.estimate_tokens() — ours is the lightweight heuristic, theirs would be model-specific.
#555 — Proactive Context Compression	Addressed: max_context_tokens + BeforeModelCallEvent hook triggers reduction before ContextWindowOverflowException.
#298 — In-event-loop cycle context management	Addressed: per_turn + compactable_after_messages + hook-based token budget checks enable within-cycle management.
#1295 — Context Limit Property on Model	Complementary: if model.context_limit ships, it could auto-configure max_context_tokens.
#1678, #1296 — Large Content Aliasing/Externalization	Related: micro-compaction replaces stale results with stubs (different strategy, same goal).
#2048 — Expose reduce_context() as Hook Event	Related: our hook calls apply_management() → reduce_context(), but doesn't fire a dedicated event.

Design Notes

• _model_call_count only increments when per_turn is enabled (preserves existing per-turn semantics)
• Summarizing manager's apply_management is an intentional no-op — proactive summarization runs exclusively via hook to prevent double-summarization with the agent's finally block
• _last_compacted_index tracks compaction progress to avoid re-scanning already-processed messages
• Hook registration in summarizing manager is guarded by max_context_tokens is not None

Test plan

• 55 new tests in test_token_aware_context_management.py covering:
  • Token estimation for all block types (text, toolResult, toolUse, image, document, video, cachePoint, guardContent)
  • Token budget enforcement via apply_management and BeforeModelCallEvent hook
  • Micro-compaction: replace/preserve/skip-already-processed/image-blocks
  • Parameter validation (max_context_tokens, compactable_after_messages)
  • _model_call_count semantics regression (not incremented when per_turn=False)
  • Integration: hook → apply_management → reduce_context full pipeline
  • _last_compacted_index adjustment after message trimming
• All 73 existing conversation manager tests pass (no regressions)
• Lint clean (ruff check), type clean (mypy)