[pull] main from danny-avila:main#117
Merged
Merged
Conversation
…#253) * fix(bedrock): gate only the tool cache point to Claude models Amazon Nova rejects a `cachePoint` under `toolConfig.tools` — `Malformed input request: #/toolConfig/tools/0: extraneous key [cachePoint] is not permitted` — yet LibreChat auto-defaults Nova to `promptCache: true`, so the SDK injected the unsupported tool checkpoint and Nova 400'd. Live probe against us.amazon.nova-lite-v1:0 (Converse): - tool cachePoint -> 400 ValidationException - message cachePoint -> 200, cacheWriteInputTokens=240 - system cachePoint -> 200, cacheWriteInputTokens=241 So only the *tool* checkpoint is Claude-only; Nova caches system/messages fine. Add `supportsBedrockToolCache(model)` and gate ONLY the tool cache point on it, leaving message/system caching enabled for Nova: - bedrock/index.ts invocationParams — skip the toolConfig cache point on non-Claude models, using a model id captured at construction (survives the application-inference-profile ARN swap during generation) - Graph.ts — skip Bedrock tool marking on non-Claude models; message/system tail cache stays gated on promptCache alone - AgentContext.ts summarization path stays gated on promptCache alone This is a capability gate (the key is rejected outright), distinct from the 1h/5m TTL value which Bedrock downgrades gracefully. Fixes danny-avila/LibreChat#13838 * fix(bedrock): treat the default Claude model as tool-cache-capable (Codex review) When `model` is omitted, LangChain initializes the model to a default Claude model, but the tool-cache gates saw '' / undefined and wrongly skipped caching for that valid Claude path. - index.ts: capture `fields?.model ?? this.model` (super() sets this.model to the default Claude model) instead of `?? ''` - Graph.ts: treat an omitted clientOptions.model as the default Claude model (only an explicit non-Claude model skips tool marking) - test: omitted model + promptCache + tools still gets the tool cache point
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )