feat(api-proxy): add allow/deny model policy to apiProxy config#4406
feat(api-proxy): add allow/deny model policy to apiProxy config#4406Copilot wants to merge 6 commits into
Conversation
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 96.61% | 96.51% | 📉 -0.10% |
| Statements | 96.51% | 96.40% | 📉 -0.11% |
| Functions | 98.70% | 98.16% | 📉 -0.54% |
| Branches | 91.03% | 90.69% | 📉 -0.34% |
📁 Per-file Coverage Changes (2 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/commands/validators/log-and-limits.ts |
90.3% → 82.7% (-7.66%) | 90.3% → 81.8% (-8.51%) |
src/config-writer.ts |
89.3% → 90.9% (+1.65%) | 89.3% → 90.9% (+1.65%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@copilot fix failing checks, also add test agentic workflow(s) |
Looked at the failing checks:
For the test agentic workflow(s) ask — could you clarify what you'd like to see?
Which would you prefer (or both)? |
zarenner
changed the title
|
@copilot resolve the merge conflicts in this pull request, merge main, recompile workflows, and make sure that all docs specs and schema are updated. nonsecurity sensitive params should be passed by stdinconfig and all security sensitive params should be passed by environment variable |
…isallowed-models # Conflicts: # containers/api-proxy/model-config.js # containers/api-proxy/model-resolver.js # containers/api-proxy/proxy-request.js
Done in 74a419a (merge) + the follow-up commit.
|
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 96.42% | 96.47% | 📈 +0.05% |
| Statements | 96.34% | 96.39% | 📈 +0.05% |
| Functions | 98.77% | 98.78% | 📈 +0.01% |
| Branches | 90.74% | 90.69% | 📉 -0.05% |
📁 Per-file Coverage Changes (2 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/config-writer.ts |
89.3% → 90.9% (+1.65%) | 89.3% → 90.9% (+1.65%) |
src/commands/validators/log-and-limits.ts |
90.3% → 92.0% (+1.68%) | 90.3% → 92.2% (+1.88%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
|
Smoke Test: API Proxy Model Allow/Deny Policy ✅ PASS
|
🔬 Smoke Test: Copilot PAT Auth — Results
Overall: FAIL — pre-computed Auth mode: PAT (COPILOT_GITHUB_TOKEN)
|
🔥 Smoke Test: PASS
PR: feat(api-proxy): add allow/deny model policy to apiProxy config
|
|
Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS Recent PRs verified:
Test Results:
PR: #4406 · Allow/deny model policy by
|
|
Smoke Test: Copilot BYOK (Direct) Mode — Azure OpenAI (Foundry, Entra / GitHub OIDC)
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "api.openai.com"See Network Configuration for more information.
|
|
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) Overall: PASS
|
|
Smoke Test Results
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
Chroot Smoke Test Results ❌
Overall: FAILED — Python and Node.js versions differ between host and chroot.
|
📡 Smoke Test: API Proxy OpenTelemetry Tracing
All tracing scenarios pass. The S3 validation script should be updated to grep
|
Smoke Test Results: GitHub Actions Services Connectivity
Overall: FAIL
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
Adds
apiProxy.allowedModelsandapiProxy.disallowedModelsto the AWF config — provider-scoped glob lists that gate which models the api-proxy will resolve and serve. Patterns follow the gh-aw model-alias spec (provider/modelglob,*/glob, or bareglob). Denylist wins; non-empty allowlist requires a match; both empty = allow all.{ "apiProxy": { "allowedModels": ["copilot/gpt-*", "copilot/claude-sonnet-*"], "disallowedModels": ["*/claude-opus-*", "copilot/o1-*"] } }Enforcement
Alias resolution —
resolveModel,selectMiddlePowerFallback, andfilterResolvableAliasesfilter candidates through the policy, so blocked models can't sneak in via aliases or middle-power fallback, and/reflecthides aliases that resolve only to blocked models.Inference requests —
rewriteModelInBodyre-checks the resolved model and returns a blocked sentinel;composeBodyTransformsshort-circuits the chain andproxy-request.jsresponds403 model_blocked_by_policy:{ "error": { "type": "model_blocked_by_policy", "message": "Model 'claude-opus-4' is blocked by AWF policy", "provider": "copilot", "model": "claude-opus-4", "reason": "disallowed" } }Plumbing
docs/awf-config.schema.json, regeneratedsrc/awf-config-schema.json),AwfFileConfigmapping,ApiProxyOptions,LogAndLimitsResultvalidation, andWrapperConfigthreading.api-proxy-service-config.tsforwardsAWF_ALLOWED_MODELS/AWF_DISALLOWED_MODELS(JSON-encodedstring[]) to the sidecar;model-config.jsparses them into a frozenMODEL_POLICYand activates the body transform whenever aliases or policy are set.Docs
docs/awf-config-spec.mdplus CLI-mapping entries.Tests
model-resolver.test.jsforparseModelGlobList,matchProviderModelPattern,checkModelPolicy, policy-awareresolveModel, and blocked-request paths throughrewriteModelInBody.validate-options.test.tsfor thevalidateModelGlobListbranch ofsrc/commands/validators/log-and-limits.ts(invalid type, non-string entries, empty/whitespace trimming, happy path).Smoke workflow
.github/workflows/smoke-model-policy.md(Copilot engine, AWF sandbox) exercises the policy end-to-end: the agentcurls$COPILOT_API_URL/chat/completionswith a model matching the injected disallow pattern and asserts HTTP 403 +"type":"model_blocked_by_policy". A post-step also greps the api-proxy JSONL logs forblocked_modelentries as a secondary check.scripts/ci/postprocess-smoke-workflows.tsnow injectsapiProxy.allowedModels: ["*"]andapiProxy.disallowedModels: ["*/awf-smoke-blocked-test-model*"]into the inlineawf-config.jsonJSON emitted by the gh-aw compiler — scoped tosmoke-model-policy.lock.ymlonly and idempotent via a negative-lookahead regex. The disallow pattern is intentionally namespaced so it never collides with a real model and the agent's own model is unaffected.scripts/ci/postprocess-smoke-workflows.test.tscovering match, replacement, both idempotency branches, and a no-match guard.