docs: comprehensive auth matrix documentation and tests by lpcox · Pull Request #4794 · github/gh-aw-firewall

lpcox · 2026-06-12T00:44:42Z

Summary

Adds comprehensive documentation and test coverage for every authentication combination supported by the api-proxy sidecar.

New files

docs/auth-matrix.md — Full specification of all auth dimensions with:
- Provider-by-provider breakdown (OpenAI, Anthropic, Copilot, Gemini)
- OIDC provider details (Azure, AWS, GCP, Anthropic WIF)
- GitHub instance type routing (github.com / GHEC / GHES)
- Coverage matrix showing tested vs gap combinations
- Links to official provider documentation
containers/api-proxy/server.auth-matrix.test.js — 46 tests covering:
- OpenAI: static key, Azure BYOK, Azure OIDC, BYOK+OIDC interaction, custom headers
- Anthropic: static key, WIF token swap (x-api-key → Authorization: Bearer), custom headers
- Copilot: GitHub token on github.com/GHEC/GHES, BYOK, GHES+BYOK interaction, Azure/GCP/AWS OIDC, target override, precedence rules
- Gemini: static key, target override, isEnabled gating
- Cross-cutting: credential isolation, isEnabled with OIDC

Key gaps now tested that weren't before

GHES token prefix — explicit test that GHES uses token (not Bearer) for GitHub OAuth
GHES + BYOK interaction — BYOK keys always use Bearer even on GHES
Copilot GCP OIDC — creates provider and injects Bearer token
Copilot AWS OIDC — creates provider, returns empty headers (SigV4 at request layer)
/models on GHES in BYOK mode — falls back to GitHub token with token prefix

Addresses #4793

Add docs/auth-matrix.md documenting every authentication combination supported by the api-proxy sidecar, including: - All 4 engines (OpenAI, Anthropic, Copilot, Gemini) - Static keys, OIDC/WIF (Azure, AWS, GCP, Anthropic native) - GitHub instance types (github.com, GHEC, GHES) - BYOK modes, custom headers, credential isolation - Links to official provider documentation Add server.auth-matrix.test.js with 46 tests covering: - Every engine × auth-type × instance-type combination - GHES token vs Bearer prefix distinction - GHES + BYOK interaction (Bearer for BYOK, token for OAuth) - OIDC token injection and isEnabled gating - Custom auth header propagation - Credential isolation behavior - /models endpoint special routing Addresses #4793 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-06-12T00:45:28Z

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 3e0810f

github-actions · 2026-06-12T00:46:16Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	96.52%	96.56%	📈 +0.04%
Statements	96.44%	96.48%	📈 +0.04%
Functions	98.78%	98.78%	➡️ +0.00%
Branches	91.12%	91.15%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/config-writer.ts`	89.9% → 91.1% (+1.19%)	89.9% → 91.1% (+1.19%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

This PR adds a single reference document that enumerates the api-proxy sidecar’s supported authentication modes across providers, and introduces a dedicated Jest suite that exercises the adapters’ auth-header behavior across those combinations to close previously identified coverage gaps.

Changes:

Add docs/auth-matrix.md documenting auth dimensions, per-provider rules, and OIDC configuration requirements.
Add containers/api-proxy/server.auth-matrix.test.js with comprehensive adapter-level tests for OpenAI/Anthropic/Copilot/Gemini auth header construction, including GHES and OIDC cases.

Show a summary per file

File	Description
docs/auth-matrix.md	New auth matrix/spec document for api-proxy behavior across providers and auth modes.
containers/api-proxy/server.auth-matrix.test.js	New Jest suite validating adapter auth headers across documented combinations (incl. GHES and OIDC).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 5

+### Custom Auth Header
+
+`AWF_OPENAI_AUTH_HEADER` overrides the header name used for the API key. The key is still sent as the header value (with `Bearer` prefix for non-Azure, without for Azure).
+


+### `/models` Endpoint (Special Case)
+
+The `/models` endpoint ALWAYS uses `COPILOT_GITHUB_TOKEN` (GitHub OAuth), never a BYOK key. This is because model listing is a GitHub platform feature, not a provider feature.
+


+| Copilot | Azure BYOK | — | ✅ | via OpenAI adapter |
+| Copilot | Azure OIDC | — | ✅ | `server.auth.test.js:749+` |
+| Copilot | AWS OIDC | — | ⚠️ partial | Scaffolding only |
+| Copilot | GCP OIDC | — | ⚠️ partial | Scaffolding only |
+| Copilot | GHES + BYOK | GHES | ⚠️ gap | Untested interaction |
+| Gemini | Static key | — | ✅ | `gemini.js:25-45` |


+JSON object of headers injected on BYOK inference requests:
+- Only active when `COPILOT_PROVIDER_API_KEY` is set
+- NOT injected on `/models` GET when GitHub OAuth token is available
+- Protected headers silently skipped with warning
+


+  it('Copilot dummy BYOK key is used as-is at the adapter level', () => {
+    // The 'dummy-byok-key-for-offline-mode' placeholder is filtered at the
+    // credential-env layer (api-proxy-credential-env.ts), not in the adapter.
+    // At the adapter level it's treated as a real key.
+    const adapter = createCopilotAdapter({


lpcox · 2026-06-12T01:41:42Z

@copilot fix this failing ci check https://github.com/github/gh-aw-firewall/actions/runs/27388545279/job/80940879241?pr=4794

Copilot · 2026-06-12T01:46:02Z

@copilot fix this failing ci check https://github.com/github/gh-aw-firewall/actions/runs/27388545279/job/80940879241?pr=4794

Fixed in 7ed6e0e. I investigated run 27388545279/job 80940879241 and the failure was Lychee returning 403 on https://platform.openai.com/docs/api-reference/authentication from docs/auth-matrix.md. I added a targeted exclusion for that URL in .github/lychee.toml so the check no longer fails on that flaky external endpoint.

github-actions · 2026-06-12T01:59:49Z

Smoke Test Results: Copilot BYOK (Direct Mode)

✅ MCP Connectivity — github-list_pull_requests verified
✅ GitHub.com Connectivity — HTTP 200
✅ File Write/Read Test — file confirmed
✅ BYOK Inference — agent → api-proxy → api.githubcopilot.com

Status: PASS — Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY)

@lpcox

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-12T01:59:53Z

🤖 Smoke Test: PASS

PR: docs: comprehensive auth matrix documentation and tests
Author: @lpcox

Test	Result
GitHub MCP connectivity	✅
github.com HTTP (200)	✅
File write/read	✅

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-12T02:00:04Z

Smoke test summary:

fix: skip symlink assertion for pre-existing path segments — ✅
refactor: extract createCounterGuard factory for guard modules — ✅
GitHub query checks — ✅
Playwright title check — ✅
File write/read check — ✅
Build (npm ci && npm run build) — ✅
Overall status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-12T02:00:59Z

🔥 Smoke Test Results — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

PR: docs: comprehensive auth matrix documentation and tests (@lpcox)

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	✅
File write/read	✅ `smoke-test-copilot-pat-27389132401.txt`

Overall: PASS

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-12T02:01:12Z

🔭 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Details
1. Module Loading	✅	`otel.js` loads cleanly; exports `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled` + test helpers
2. Test Suite	✅	39/39 tests passed in `otel.test.js` (0 failures)
3. Env Var Forwarding	✅	`api-proxy-service-config.ts` forwards `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`
4. Token Tracker Integration	✅	`onUsage` callback present in `token-tracker-http.js` (invoked at line 255)
5. OTEL Diagnostics	✅	No OTLP endpoint set → `FileSpanExporter` fallback active (writes to `/var/log/api-proxy/otel.jsonl`); graceful degradation confirmed

All scenarios passed. Module auto-initializes on load (_init() called at require-time); isEnabled() returns true with file-based fallback when no OTLP endpoint is configured.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-12T02:01:28Z

Chroot Runtime Version Comparison

Runtime	Host Version	Chroot Version	Match?
Python	3.12.13	3.12.3	❌
Node.js	v24.16.0	v22.22.3	❌
Go	go1.22.12	go1.22.12	✅

Result: Not all tests passed. Go matches, but Python and Node.js versions differ between host and chroot.

Tested by Smoke Chroot

github-actions · 2026-06-12T02:02:12Z

@lpcox
GitHub MCP connectivity: ✅
GitHub.com connectivity: ✅
File write/read test: ✅
Inference BYOK path: ✅
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

github-actions · 2026-06-12T02:02:13Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Note: Java required -Dmaven.repo.local=/tmp/gh-aw/agent/m2-repo workaround because ~/.m2/ is owned by root on this runner, preventing Maven from creating the default local repository directory.

Generated by Build Test Suite for issue #4794 · ◷

github-actions · 2026-06-12T02:02:57Z

Smoke Test: GitHub Actions Services Connectivity

Check	Result
Redis PING	❌ timeout
PostgreSQL pg_isready	❌ no response
PostgreSQL SELECT 1	❌ timeout

Overall: FAIL — host.docker.internal is not reachable from this runner; all service containers are unreachable.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-12T02:07:29Z

GitHub API: ✅ PASS
GitHub check: ✅ PASS
File verify: ✅ PASS

Total: PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-06-13T01:38:15Z

@lpcox
docs: comprehensive auth matrix documentation and tests
✅ GitHub MCP ✅ GitHub.com ✅ File I/O ✅ BYOK Inference
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

Copilot AI review requested due to automatic review settings June 12, 2026 00:44

Copilot started reviewing on behalf of lpcox June 12, 2026 00:45 View session

lpcox temporarily deployed to aoai-model June 12, 2026 00:47 — with GitHub Actions Inactive

Copilot AI reviewed Jun 12, 2026

View reviewed changes

github-actions Bot mentioned this pull request Jun 12, 2026

[aw] Contribution Check failed #4772

Closed