docs: document Anthropic structured-output schema complexity limits (#682)#686
Open
tombeckenham wants to merge 4 commits into
Conversation
Add `structuredOutput: 'auto' | 'native' | 'tool'` to `chat()` (default 'auto'). When a provider rejects a large structured-output schema (Anthropic "compiled grammar is too large", directly or for anthropic/* via OpenRouter), the engine transparently retries via the lenient forced-tool path instead of hard-failing. Detection is delegated to a new optional adapter predicate `isStructuredOutputSchemaError`; @tanstack/ai-anthropic and @tanstack/ai-openrouter implement it, and OpenRouter gains a forced-tool structured-output mode. Closes #682. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughAdded a new "Anthropic schema complexity limits" subsection to the structured outputs overview documentation. The subsection describes how Anthropic rejects oversized schemas with compilation-size errors, lists common problematic schema constructs, and directs users to simplification strategies and official documentation. ChangesAnthropic schema complexity limits documentation
🎯 1 (Trivial) | ⏱️ ~3 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 3❌ Failed checks (2 warnings, 1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version PreviewNo changeset entries found. Merging this PR will not cause a version bump for any packages. |
|
View your CI Pipeline Execution ↗ for commit dcbc2ac
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
packages/ai-openrouter/tests/openrouter-adapter.test.ts (1)
2563-2568: 💤 Low valueConsider adding
undefinedto negative test cases for consistency.The Anthropic adapter tests check both
nullandundefined(line 1379-1380 in anthropic-adapter.test.ts), but this test only checksnull. For completeness, consider adding:expect(adapter.isStructuredOutputSchemaError(undefined)).toBe(false)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-openrouter/tests/openrouter-adapter.test.ts` around lines 2563 - 2568, Add a negative test asserting undefined is treated like null: update the test that calls adapter.isStructuredOutputSchemaError(...) to also include expect(adapter.isStructuredOutputSchemaError(undefined)).toBe(false) so the spec covers both null and undefined cases for the isStructuredOutputSchemaError function.packages/ai-openrouter/src/adapters/text.ts (1)
787-793: 💤 Low valueMissing
contentfield inTEXT_MESSAGE_CONTENTevent.The
TEXT_MESSAGE_CONTENTevent typically includes bothdelta(the incremental content) andcontent(the accumulated content). Compare with line 502 instructuredOutputStreamwhich includes both fields. Since this is a single-shot emission, they would be identical, but omittingcontentmay cause consumers expecting the accumulated field to seeundefined.Suggested fix
yield { type: EventType.TEXT_MESSAGE_CONTENT, messageId, delta: result.rawText, + content: result.rawText, model, timestamp, }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-openrouter/src/adapters/text.ts` around lines 787 - 793, The TEXT_MESSAGE_CONTENT event emission is missing the accumulated "content" field; update the yield that returns type EventType.TEXT_MESSAGE_CONTENT (the block using messageId, delta: result.rawText, model, timestamp) to include content (set it to the same accumulated text, e.g., result.rawText for this single-shot case) so consumers expecting both delta and content receive a defined value (follow the same pattern used in structuredOutputStream where both delta and content are provided).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@testing/e2e/global-setup.ts`:
- Around line 107-130: The comment claiming a malformed body is treated as a
native attempt is incorrect because on JSON.parse failure `body` stays {} so
`body.output_config` is undefined and the request is not rejected; update either
the comment or the behavior: either change the comment above the try/catch to
note that parse failures fall through (so the `if (body.output_config != null)`
branch only triggers when output_config exists), or explicitly reject on parse
failure by detecting the parse error and returning the same 400 JSON error (use
the existing GRAMMAR_TOO_LARGE_MESSAGE and the same response logic used in the
`if (body.output_config != null)` block); refer to readBody, the parsed variable
`body`/`body.output_config`, and GRAMMAR_TOO_LARGE_MESSAGE to implement the
chosen fix.
---
Nitpick comments:
In `@packages/ai-openrouter/src/adapters/text.ts`:
- Around line 787-793: The TEXT_MESSAGE_CONTENT event emission is missing the
accumulated "content" field; update the yield that returns type
EventType.TEXT_MESSAGE_CONTENT (the block using messageId, delta:
result.rawText, model, timestamp) to include content (set it to the same
accumulated text, e.g., result.rawText for this single-shot case) so consumers
expecting both delta and content receive a defined value (follow the same
pattern used in structuredOutputStream where both delta and content are
provided).
In `@packages/ai-openrouter/tests/openrouter-adapter.test.ts`:
- Around line 2563-2568: Add a negative test asserting undefined is treated like
null: update the test that calls adapter.isStructuredOutputSchemaError(...) to
also include
expect(adapter.isStructuredOutputSchemaError(undefined)).toBe(false) so the spec
covers both null and undefined cases for the isStructuredOutputSchemaError
function.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9504d444-b210-4ea1-80a8-a39d0f5244ac
📒 Files selected for processing (18)
.changeset/structured-output-schema-fallback.mddocs/structured-outputs/overview.mddocs/structured-outputs/streaming.mdpackages/ai-anthropic/src/adapters/text.tspackages/ai-anthropic/tests/anthropic-adapter.test.tspackages/ai-openrouter/src/adapters/text.tspackages/ai-openrouter/tests/openrouter-adapter.test.tspackages/ai/skills/ai-core/adapter-configuration/SKILL.mdpackages/ai/skills/ai-core/structured-outputs/SKILL.mdpackages/ai/src/activities/chat/adapter.tspackages/ai/src/activities/chat/index.tspackages/ai/tests/chat-structured-output-fallback.test.tspackages/ai/tests/test-utils.tstesting/e2e/fixtures/structured-output-fallback/openrouter-tool.jsontesting/e2e/global-setup.tstesting/e2e/src/routeTree.gen.tstesting/e2e/src/routes/api.structured-output-fallback.tstesting/e2e/tests/structured-output-fallback.spec.ts
tombeckenham
commented
Jun 3, 2026
Contributor
Author
tombeckenham
left a comment
tombeckenham
left a comment
There was a problem hiding this comment.
Note, this should adhere to https://platform.claude.com/docs/en/build-with-claude/structured-outputs#schema-complexity-limits
…tion Widen the structured-output schema-rejection predicate beyond the observed "compiled grammar is too large" message to also match Anthropic's canonical 400 message "Schema is too complex for compilation", so `structuredOutput: 'auto'` falls back on the documented error too. Applies to both @tanstack/ai-anthropic and @tanstack/ai-openrouter. Document the boundary in the structured-outputs overview and skill: detection is reactive (we never pre-check schemas against Anthropic's caps), `'auto'` recovers from grammar-size rejections only, and the numeric limits are linked out rather than copied (they change and aren't all published). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/ai-openrouter/src/adapters/text.ts (1)
1602-1608:⚠️ Potential issue | 🟠 Major | ⚡ Quick winNarrow the generic schema-path match.
Line 1607 currently treats any error mentioning
output_config.format.schemaas retryable. That will also catch non-recoverable request-validation errors for an invalid schema, sostructuredOutput: 'auto'can silently rerun through the lenient tool path instead of surfacing the real caller bug. The core contract inpackages/ai/src/activities/chat/adapter.tsonly allowstruefor recoverable schema-compilation rejections.Suggested fix
function isOpenRouterStructuredOutputSchemaError(error: unknown): boolean { const text = collectErrorText(error).toLowerCase() return ( text.includes('compiled grammar is too large') || - text.includes('schema is too complex for compilation') || - text.includes('output_config.format.schema') + text.includes('schema is too complex for compilation') ) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-openrouter/src/adapters/text.ts` around lines 1602 - 1608, The function isOpenRouterStructuredOutputSchemaError currently treats any error mentioning "output_config.format.schema" as recoverable; narrow this by only returning true when the error text contains "output_config.format.schema" together with compilation-related terms (e.g., "compile", "compilation", "compiled", "too large", "too complex", or similar) to ensure only schema-compilation failures are considered retryable; update the logic in isOpenRouterStructuredOutputSchemaError to check both the presence of "output_config.format.schema" AND at least one compilation-related keyword (instead of matching "output_config.format.schema" alone).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@packages/ai-openrouter/src/adapters/text.ts`:
- Around line 1602-1608: The function isOpenRouterStructuredOutputSchemaError
currently treats any error mentioning "output_config.format.schema" as
recoverable; narrow this by only returning true when the error text contains
"output_config.format.schema" together with compilation-related terms (e.g.,
"compile", "compilation", "compiled", "too large", "too complex", or similar) to
ensure only schema-compilation failures are considered retryable; update the
logic in isOpenRouterStructuredOutputSchemaError to check both the presence of
"output_config.format.schema" AND at least one compilation-related keyword
(instead of matching "output_config.format.schema" alone).
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 3253bff2-76b6-4daa-8f9c-f2494ac93f36
📒 Files selected for processing (7)
.changeset/structured-output-schema-fallback.mddocs/structured-outputs/overview.mdpackages/ai-anthropic/src/adapters/text.tspackages/ai-anthropic/tests/anthropic-adapter.test.tspackages/ai-openrouter/src/adapters/text.tspackages/ai-openrouter/tests/openrouter-adapter.test.tspackages/ai/skills/ai-core/structured-outputs/SKILL.md
✅ Files skipped from review due to trivial changes (4)
- docs/structured-outputs/overview.md
- packages/ai/skills/ai-core/structured-outputs/SKILL.md
- .changeset/structured-output-schema-fallback.md
- packages/ai-anthropic/tests/anthropic-adapter.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/ai-anthropic/src/adapters/text.ts
- packages/ai-openrouter/tests/openrouter-adapter.test.ts
Reverts the structuredOutput 'auto'/'native'/'tool' strategy option, the forced-tool fallback retry, the isStructuredOutputSchemaError adapter predicate (Anthropic + OpenRouter grammar-message detection), and OpenRouter's forced-tool structured-output mode. Transparently falling back to the lenient tool path hides schema problems users should fix at the source — Anthropic's grammar limits are a signal the schema is too complex, and simplifying it (fewer optionals, no catch/default wrappers) is the right fix. A docs note replaces the machinery (#682). This reverts commits a07718d and bc43c30. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…682) Large/complex schemas are rejected by Anthropic's grammar compiler ("Schema is too complex for compilation" / "compiled grammar is too large"), directly and for anthropic/* models via OpenRouter. Document the errors, the common schema constructs that trigger them (optionals, catch/default wrappers, unions, unconstrained strings), and link to Anthropic's docs for current limits rather than copying numbers that change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tombeckenham
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Closes #682.
chat({ outputSchema })routes Claude 4.5+ structured output through Anthropic's nativeoutput_config; a large/complex schema is rejected with "The compiled grammar is too large" / "Schema is too complex for compilation" (also reproducible foranthropic/*models via OpenRouter).This PR originally added a
structuredOutput: 'auto' | 'native' | 'tool'option that transparently re-ran the request through a lenient forced-tool path on schema rejection. That fallback has been removed — the PR is now docs-only: a new "Anthropic schema complexity limits" section in the structured-outputs overview that documents both rejection messages, the schema constructs that commonly trigger them (.optional()fields,.catch()/.default()wrappers, unions, deep nesting, unconstrained optional strings), and links to Anthropic's docs for the current limits.Why not a fallback?
.catch()wrappers and unneeded optional strings).input_schema, so the "schema-constrained output" the user asked for is quietly replaced with a weaker contract — different failure modes, no grammar enforcement — without the user opting in.isStructuredOutputSchemaErroradapter predicate, error-message sniffing in two adapters, a forced-tool mode for OpenRouter, and a retry/replay engine path — all keyed off matching provider error strings that can change at any time.Surfacing the provider error unchanged plus documenting how to simplify the schema is the better trade.
✅ Checklist
pnpm run test:pr.🚀 Release Impact
🤖 Generated with Claude Code