Skip to content

feat(mcp): add compact description mode to reduce context token overhead#2853

Open
lg320531124 wants to merge 2 commits into
volcengine:mainfrom
lg320531124:feat/mcp-compact-description
Open

feat(mcp): add compact description mode to reduce context token overhead#2853
lg320531124 wants to merge 2 commits into
volcengine:mainfrom
lg320531124:feat/mcp-compact-description

Conversation

@lg320531124

@lg320531124 lg320531124 commented Jun 26, 2026

Copy link
Copy Markdown

Problem

OpenViking MCP server exposes 15 tools whose description docstrings are injected into the LLM context on every turn. These descriptions total ~6000 chars (~1500 tokens), consuming context budget even when the model only needs 2-3 tools per turn.

Fix

Add a compact description mode that truncates each tool description to its first sentence. Controlled via OPENVIKING_MCP_DESCRIPTION_MODE env var:

Mode Behavior
full (default) No change — full docstrings
compact Truncate each description to the first sentence

Savings

Tool Full chars Compact chars Savings
find ~126 ~48 62%
search ~152 ~74 51%
read ~157 ~55 65%
ls ~99 ~62 37%
remember ~159 ~51 68%
add_resource ~2120 ~29 99%
list_watches ~239 ~201 16%
cancel_watch ~243 ~105 57%
grep ~195 ~67 66%
glob ~147 ~50 66%
forget ~348 ~51 85%
code_outline ~610 ~108 82%
code_search ~824 ~142 83%
code_expand ~574 ~390 32%
health ~47 ~47 0%
Total ~6040 ~1480 75%

Usage

# Enable compact descriptions
export OPENVIKING_MCP_DESCRIPTION_MODE=compact

# Default (full descriptions)
# export OPENVIKING_MCP_DESCRIPTION_MODE=full  # or just unset it

Implementation

Wraps mcp.tool decorator to auto-compact fn.__doc__ before registration. Zero changes to tool logic — only affects the description string passed to the MCP schema.

Fixes #2849

Add OPENVIKING_MCP_DESCRIPTION_MODE environment variable:
- 'full' (default): no change, full docstrings exposed as tool descriptions
- 'compact': truncate each tool description to the first sentence

When 15 MCP tools are exposed, their combined descriptions consume
~200-400 context tokens per turn. Compact mode reduces this by 33-60%
by keeping only the first sentence of each docstring.

Implementation wraps the @mcp.tool decorator to auto-compact
descriptions at registration time. No changes to individual tool
functions.

Set via: OPENVIKING_MCP_DESCRIPTION_MODE=compact ov serve

Fixes volcengine#2849
@ZaynJarvis

Copy link
Copy Markdown
Collaborator

looks okay, @t0saki pls check for effect changes. (first line should be the one-liner doc in code. but i don't see multi-line in docs, probably schemas?)

if lost schema would that affect MCP server's behavior and result in another round of tool usage search

@lg320531124

Copy link
Copy Markdown
Author

Thanks for the review @ZaynJarvis!

Re: "first line should be the one-liner doc" — Agreed. The current docstrings follow Python convention (one-line summary + blank line + details), so _compact_description naturally keeps just the summary line. This is exactly the intended behavior.

Re: "if lost schema would that affect MCP server behavior and result in another round of tool usage search" — Good concern. Two points:

  1. Schema is not affected. The compact mode only truncates the description string in the MCP tool definition. The inputSchema (parameter names, types, required/optional, enums) is untouched — the LLM still sees the full parameter signature and can construct correct calls.

  2. Description vs Schema separation. In MCP, description helps the LLM choose the right tool; inputSchema helps it use the tool correctly. Compact mode keeps the first sentence (which states what the tool does), so tool selection accuracy is preserved. What gets truncated is usage guidance like "Use only after code_outline" or "For directory listing, use the list tool instead" — these are helpful but not required for correct invocation.

That said, if you prefer a more conservative approach, I can change the truncation to keep the first paragraph (up to \n\n) instead of the first sentence (up to . ). This would preserve more context for tools like code_expand and forget that have important usage warnings in their second sentence. Let me know which you prefer.

@t0saki

t0saki commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

I think 6k total tokens is acceptable, but I do agree some tools desc can be compressed, especially add_resource and code series. I worry the results will deteriorate, hope we can shorten the description length of add_resource and the code series (can make direct changes, no need an extra configuration).

…escriptions

Instead of OPENVIKING_MCP_DESCRIPTION_MODE=compact config switch,
directly shorten tool descriptions that were unnecessarily verbose.
Add_resource dropped from 2120→191 chars, code_search 824→247,
code_outline 610→158, code_expand 432→155, forget 348→118.

Total description chars: ~4700→2078 (56% reduction).

Per @t0saki review feedback on volcengine#2853.
@lg320531124

Copy link
Copy Markdown
Author

@t0saki — good point, agree that a config switch is over-engineering for this. Rewrote in 8f1d5b1:

  • Removed the entire OPENVIKING_MCP_DESCRIPTION_MODE mechanism
  • Directly shortened verbose tool descriptions instead:
    • add_resource: 2120→191 chars
    • code_search: 824→247 chars
    • code_outline: 610→158 chars
    • code_expand: 432→155 chars
    • forget: 348→118 chars
    • cancel_watch / list_watches: also trimmed
  • Total description chars: ~4700→2078 (56% reduction)

No env var, no config — just shorter descriptions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

[Feature] MCP tool schema descriptions too verbose — add compact mode to reduce context token overhead

3 participants