feat(ai-providers): add Apple Intelligence on-device provider for macOS 26+#1554
feat(ai-providers): add Apple Intelligence on-device provider for macOS 26+#1554datlechin wants to merge 7 commits into
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3667d72492
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| static func makeAppleIntelligenceProvider() -> ChatTransport { | ||
| let status = AppleIntelligenceAvailability.currentStatus() | ||
| if #available(macOS 26, *), status == .available { | ||
| return AppleIntelligenceTransport() |
There was a problem hiding this comment.
When this target is built with an SDK where canImport(FoundationModels) is false, AppleIntelligenceTransport is compiled out by the #if in its own file, but this factory still references it. The weak-link flags do not help when the SDK module is unavailable, so pre-Xcode 26 builds fail instead of falling back to UnavailableTransport; wrap this branch in #if canImport(FoundationModels) and return the unavailable transport otherwise.
Useful? React with 👍 / 👎.
…nd share the System Settings opener
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 13e297c8e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if #available(macOS 26, *), status == .available { | ||
| return AppleIntelligenceTransport() | ||
| } | ||
| return UnavailableTransport(reason: status.statusText) |
There was a problem hiding this comment.
If an existing Apple Intelligence provider is resolved while currentStatus() is transiently unavailable, such as .modelNotReady during the model download, this branch returns an UnavailableTransport; createProvider then caches that transport for the config, so later chats keep failing even after the model becomes available unless the app restarts or the provider settings change. Make this status-dependent fallback bypass the provider cache or invalidate it when availability changes.
Useful? React with 👍 / 👎.
# Conflicts: # CHANGELOG.md
…ligence-transport # Conflicts: # CHANGELOG.md # TablePro/Models/AI/AIModels.swift # TablePro/Views/Settings/AIProviderDetailSheet.swift # TablePro/Views/Settings/AISettingsView.swift # docs/features/ai-assistant.mdx
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d58ce345f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| internal static func seedAppleIntelligenceIfEligible(_ settings: AISettings) -> AISettings { | ||
| guard settings.providers.isEmpty else { return settings } | ||
| guard AppleIntelligenceAvailability.currentStatus() == .available else { return settings } |
There was a problem hiding this comment.
When Apple Intelligence is available and the user removes its provider, the saved AI settings become empty; on the next launch this same guard treats those empty settings as a fresh install and silently re-adds Apple Intelligence. Since the row exposes a Remove action, removal should be durable, e.g. by seeding only on a true first-run/migration marker or by recording that the seeded provider was dismissed.
Useful? React with 👍 / 👎.
| // MARK: - AI Provider Type | ||
|
|
||
| enum AIProviderType: String, Codable, CaseIterable, Identifiable, Sendable { | ||
| case appleIntelligence |
There was a problem hiding this comment.
Because AISettingsView.orderedAddableTypes iterates AIProviderType.allCases and only filters .custom, adding this new CaseIterable case also makes Apple Intelligence appear under “Add Provider…”. Choosing it can create additional Apple provider configs; the provider list then filters Apple configs out except for first(where:), so duplicates can become hidden/uneditable and an unavailable Apple provider can be saved via the detail sheet despite the pinned row being disabled.
Useful? React with 👍 / 👎.
| let text = turn.plainText | ||
| guard !text.isEmpty else { continue } |
There was a problem hiding this comment.
For Apple Intelligence follow-up turns after a tool call, the previous tool output is stored as a .toolResult block in a user turn, but this transcript builder derives only plainText and skips empty text turns. In that scenario the database/query result that the model just used is dropped from the next request's history, so follow-up questions like “use that result” lose the actual tool output unless the assistant happened to restate it in prose.
Useful? React with 👍 / 👎.
…on error instead of a cryptic code
… so tool calls don't fail mid-stream
…w a readable message instead of a raw code
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 193374d9c2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| appleIntelligenceStatus = AppleIntelligenceAvailability.currentStatus() | ||
| while appleIntelligenceStatus == .modelNotReady { | ||
| try? await Task.sleep(for: .seconds(15)) | ||
| if Task.isCancelled { break } | ||
| appleIntelligenceStatus = AppleIntelligenceAvailability.currentStatus() | ||
| } |
There was a problem hiding this comment.
When Apple Intelligence is initially .notEnabled, this task records that status once and then stops polling because the loop only handles .modelNotReady. In the Settings row flow I inspected, tapping the row opens System Settings, but after the user enables Apple Intelligence the cached appleIntelligenceStatus remains .notEnabled, so tapping the row keeps reopening System Settings instead of adding the provider until the AI settings view is recreated.
Useful? React with 👍 / 👎.
|
Closing this. After getting the on-device transport working end to end, Foundation Models isn't a good fit for TablePro's agentic AI chat, and the blocker is structural rather than a bug we can patch. What worksPlain chat and single-turn tool calls run fine. Sign-in and availability detection, the transcript building, schema generation, and the tool bridge all work on macOS 26. Why it can't shipThe on-device model has a fixed ~4k-token context window. TablePro's AI chat sends a large system prompt plus every tool's JSON schema, and each tool result (for example Making it usable would mean trimming conversation history and truncating tool outputs for this provider only. That degrades answers: the model would see partial results and forget earlier turns. It isn't a tradeoff worth shipping when every other provider runs the same agent flow without it. Worth keeping for a future attemptTwo findings from this work (the code stays on the
Worth revisiting if a future macOS release grows the on-device context window, or if we add a context-condensation strategy for small-window models. Reopening then. |
1 participant
Closes #1048.
Adds Apple Intelligence as an on-device AI chat provider on macOS 26 and later, backed by the Foundation Models framework. No API key, no network: schema and queries stay on the Mac. The app's macOS 14.0 minimum does not change; everything is gated behind
#available(macOS 26, *)andFoundationModelsis weak-linked.What it does
AppleIntelligenceTransport: ChatTransportoverLanguageModelSession. It flows through the existing registry, factory, and transport pipeline as a realAIProviderConfig(type: .appleIntelligence, new.deviceauth style), so the model picker, per-turn attribution, and persistence work with no special-casing.How the hard parts work
streamResponseyields cumulative snapshots, so the transport diffs each against the previous (with a prefix guard) to emit text deltas.Tool.ArgumentsisGeneratedContentand theGenerationSchemais built at runtime from ourJsonValueviaDynamicGenerationSchema. The framework runs the tool loop itself; eachTool.callbridges to the existing approval and execution path throughtoolInvocationRequestand a reply token.AppleIntelligenceStatusfacade wraps the#availableandSystemLanguageModel.default.availabilitycheck so the un-gated UI can read availability without importing the framework.Transcriptfrom the conversation each call, so it needs no reset wiring.The generic Copilot tool-dispatch functions were renamed to provider-neutral names since this transport reuses them.
Tests
New tests cover the availability facade mapping, the schema bridge, seeding idempotency and no-override, and the factory guard (an Apple Intelligence config never falls back to an OpenAI-compatible transport). Foundation Models tests are gated to macOS 26.
Before merge
Transcriptinitializers inAppleIntelligenceTransport.buildTranscriptwere written against Apple's docs but not compiled here.com.apple.Siri-Settings.extension) is inferred; it falls back to the Settings root if the identifier is wrong. Worth confirming on a real macOS 26 machine.