feat(orchestration): add content filtering and prompt shield module

[lenin-ribeiro]

Disclaimer: Do not include SAP-internal or customer-specific information in this PR (e.g. internal system URLs, customer names, tenant IDs, or confidential configurations). This is a public repository.

Description

Adds content filtering and prompt-shield support to the existing sap_cloud_sdk.aicore module. Azure Content Safety filtering and prompt-attack detection are activated automatically for every SAP AI Core model call made through LiteLLM.

Filtering is enabled by default when set_aicore_config() is called — no code change is required by the developer. The default policy blocks medium+ severity (Severity.MEDIUM) on every category and enables Prompt Shield on input for all sap/* model calls.

How it works

set_aicore_config() calls set_filtering() at the end, which patches litellm.GenAIHubOrchestrationConfig with a subclass (FilteringOrchestrationConfig) that:

• Injects modules.filtering (Azure Content Safety config) into every v2 completion request body via transform_request
• Detects filter rejections in responses and raises ContentFilteredError via transform_response
• Unwraps filter rejections embedded in LiteLLM APIConnectionError exceptions via extract_filter_blocked()

LiteLLM still makes the HTTP call and Traceloop/OTel instrumentation is fully preserved.

Developer experience

Zero code change for the common case — existing agent code is unchanged:

from sap_cloud_sdk.aicore import set_aicore_config
set_aicore_config()
# ← filtering is now active at Severity.MEDIUM on all categories + prompt_shield=True

Thresholds configurable via env vars (set before set_aicore_config()):

AICORE_FILTER_SELF_HARM=0    # strict — block any detected self-harm content
AICORE_FILTER_ENABLED=false  # disable filtering entirely

Programmatic override at runtime uses the class-based API (mirrors the gen_ai_hub.orchestration.models.content_filtering shape):

from sap_cloud_sdk.aicore import (
    AzureContentFilter,
    ContentFiltering,
    InputFiltering,
    OutputFiltering,
    Severity,
    set_filtering,
)

set_filtering(ContentFiltering(
    input_filtering=InputFiltering(filters=[
        AzureContentFilter(
            hate=Severity.STRICT,
            violence=Severity.STRICT,
            sexual=Severity.STRICT,
            self_harm=Severity.STRICT,
            prompt_shield=True,
        ),
    ]),
    output_filtering=OutputFiltering(filters=[
        AzureContentFilter(
            hate=Severity.MEDIUM,
            violence=Severity.MEDIUM,
            sexual=Severity.MEDIUM,
            self_harm=Severity.MEDIUM,
        ),
    ]),
))

A direction can stack multiple filter providers — AzureContentFilter plus LlamaGuard38bFilter:

from sap_cloud_sdk.aicore import LlamaGuard38bFilter

set_filtering(ContentFiltering(
    input_filtering=InputFiltering(filters=[
        AzureContentFilter(prompt_shield=True),
        LlamaGuard38bFilter(hate=True, violent_crimes=True),
    ]),
))

Disable filtering at runtime:

from sap_cloud_sdk.aicore import disable_filtering

disable_filtering()

Handling blocked requests:

from sap_cloud_sdk.aicore import ContentFilteredError, extract_filter_blocked

try:
    result = await llm.ainvoke(messages)
except ContentFilteredError as e:
    return "Your request was blocked by content safety policy."
except Exception as e:
    if blocked := extract_filter_blocked(e):
        return "Your request was blocked by content safety policy."
    raise

Related Issue

N/A — new feature proposed and implemented by the App Foundation agent team.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Documentation update
• Code refactoring
• Dependency update

How to Test

Run the unit tests:

uv run python -m pytest tests/aicore/ tests/core/unit/ -v
# Expected: 749 passed

Verify auto-activation:

import litellm
from sap_cloud_sdk.aicore import set_aicore_config
from sap_cloud_sdk.aicore.filtering._litellm_patch import FilteringOrchestrationConfig

set_aicore_config()
assert litellm.GenAIHubOrchestrationConfig is FilteringOrchestrationConfig
print("Filtering activated ✓")

Verify wire format:

from sap_cloud_sdk.aicore import (
    AzureContentFilter, ContentFiltering, InputFiltering, Severity,
)

cfg = ContentFiltering(
    input_filtering=InputFiltering(filters=[
        AzureContentFilter(self_harm=Severity.STRICT, prompt_shield=True),
    ]),
)
d = cfg.to_dict()
assert "input" in d
assert d["input"]["filters"][0]["type"] == "azure_content_safety"
assert d["input"]["filters"][0]["config"]["self_harm"] == 0
assert d["input"]["filters"][0]["config"]["prompt_shield"] is True
print("Wire format correct ✓")

Integration tests against a live AI Core deployment (skip when env absent):

# Requires AICORE_CLIENT_ID, AICORE_CLIENT_SECRET, AICORE_AUTH_URL, AICORE_BASE_URL,
# AICORE_RESOURCE_GROUP, AICORE_FILTER_TEST_MODEL, AICORE_FILTER_TEST_SELF_HARM_PROMPT
# set (or in .env_integration_tests).
uv run python -m pytest tests/aicore/integration/ -v

Checklist

• I have read the Contributing Guidelines
• I have verified that my changes solve the issue
• I have added/updated automated tests to cover my changes (68 filtering unit tests + 22 core/env tests + 4 integration scenarios)
• All unit tests pass locally
• I have verified that my code follows the Code Guidelines
• I have updated documentation (aicore/user-guide.md)
• I have added type hints for all public APIs
• My code does not contain sensitive information (credentials, tokens, etc.)
• I have followed Conventional Commits for commit messages

Additional Notes

Public API surface (sap_cloud_sdk.aicore)

Symbol	Purpose
set_aicore_config	Load AI Core credentials and activate filtering (existing — now with side effect)
set_filtering	Install a ContentFiltering configuration (or re-apply env defaults with no args)
disable_filtering	Turn filtering off at runtime
ContentFiltering	Top-level container with input_filtering + output_filtering
InputFiltering	Direction container — accepts a list of filters
OutputFiltering	Direction container — accepts a list of filters + optional stream_options
AzureContentFilter	Azure Content Safety provider — 4 severity categories + prompt_shield
LlamaGuard38bFilter	Llama Guard 3 8B provider — 14 boolean category flags
ContentFilter	Abstract base — exposed for type hints / custom providers
Severity	IntEnum of STRICT/LOW/MEDIUM/OFF (0/2/4/6)
ContentFilteredError	Raised on filter rejection (direction, details, request_id)
OrchestrationError	Base class for filter-related exceptions
extract_filter_blocked	Unwrap input-filter rejections from LiteLLM APIConnectionError

All names available as from sap_cloud_sdk.aicore import … (flat).

New files

File	Purpose
src/sap_cloud_sdk/core/env.py	Typed env-var readers: read_env_str, read_env_bool, read_env_choice
src/sap_cloud_sdk/aicore/filtering/__init__.py	Public filtering API: set_filtering, disable_filtering, re-exports
src/sap_cloud_sdk/aicore/filtering/_filters.py	ContentFilter base, AzureContentFilter, LlamaGuard38bFilter
src/sap_cloud_sdk/aicore/filtering/_modules.py	InputFiltering, OutputFiltering, ContentFiltering (with from_env())
src/sap_cloud_sdk/aicore/filtering/_models.py	Severity enum
src/sap_cloud_sdk/aicore/filtering/_litellm_patch.py	FilteringOrchestrationConfig subclass, _install(), extract_filter_blocked()
src/sap_cloud_sdk/aicore/filtering/exceptions.py	ContentFilteredError, OrchestrationError
tests/core/unit/test_env.py	22 cases covering env-helper defaults, parsing, validation
tests/aicore/filtering/unit/test_filters.py	22 provider-class tests
tests/aicore/filtering/unit/test_modules.py	18 direction-container + from_env() tests
tests/aicore/filtering/unit/test_patch.py	15 LiteLLM patch + response-detection tests
tests/aicore/filtering/unit/test_filtering.py	10 set_filtering / disable_filtering behaviour tests
tests/aicore/filtering/unit/test_models.py	3 Severity enum tests
tests/aicore/integration/{conftest.py,filtering.feature,test_filtering_bdd.py}	4 live BDD scenarios — OFF baseline, ON benign, input filter STRICT, Prompt Shield jailbreak

Env vars reference

Variable	Default	Description
AICORE_FILTER_ENABLED	true	Set false to disable filtering entirely
AICORE_FILTER_DIRECTIONS	input,output	Which sides to filter
AICORE_FILTER_HATE	4	Azure severity 0/2/4/6
AICORE_FILTER_VIOLENCE	4	Azure severity 0/2/4/6
AICORE_FILTER_SEXUAL	4	Azure severity 0/2/4/6
AICORE_FILTER_SELF_HARM	4	Azure severity 0/2/4/6
AICORE_FILTER_PROMPT_SHIELD	true	Input-only jailbreak detection
AICORE_FILTER_TEST_MODEL	n/a	Model name used by the integration test suite (CI/operator)
AICORE_FILTER_TEST_SELF_HARM_PROMPT	n/a	Self-harm test prompt used by integration tests; kept out of source

Telemetry

Module.AICORE records four operations: set_aicore_config, set_filtering, disable_filtering, extract_filter_blocked. The previous Module.ORCHESTRATION was dropped — orchestration is folded into AI Core as a sub-package.

Restructure highlights (post-review)

This PR was substantially restructured in response to review feedback from @NicoleMGomes:

• chore(deps): update traceloop-sdk requirement from ~=0.52.0 to >=0.52,<0.54 #1 Folded sap_cloud_sdk.orchestration under sap_cloud_sdk.aicore as the filtering sub-package. Public API now imported flat from sap_cloud_sdk.aicore.
• chore(deps): update hatchling requirement from ~=1.27.0 to >=1.27,<1.30 #2 Updated set_aicore_config docstring to describe the filtering side effect.
• chore(deps): update opentelemetry-exporter-otlp-proto-grpc requirement from ~=1.38.0 to >=1.38,<1.41 #3 Removed the lazy import / try-except block in set_aicore_config — no longer a circular-dependency risk, and any activation failure now surfaces rather than being swallowed.
• chore(deps): update setuptools requirement from ~=80.9.0 to >=80.9,<82.1 #4 Telemetry consolidated under Module.AICORE; Module.ORCHESTRATION dropped.
• chore(deps): update requests requirement from ~=2.31.0 to >=2.31,<2.33 #5 set_filtering(enabled=False) replaced with dedicated disable_filtering() function.
• feat: context propagation and operation type attributes #6 Added @record_metrics decorator to extract_filter_blocked for consistent public-method telemetry.
• docs: prepare documentation for open source release #7 Added typed-env helpers (read_env_str, read_env_bool, read_env_choice) in src/sap_cloud_sdk/core/env.py. secret_resolver was confirmed to be a credential loader (mount-with-env-fallback for dataclass strings), not appropriate for runtime feature toggles.
• chore: version bump #8 Literal[0, 2, 4, 6] replaced by Severity(IntEnum) (STRICT/LOW/MEDIUM/OFF) in all public signatures and dataclass fields.
• chore: apply pre-commit hooks across all files #9 Removed the README breaking-change callout; the aicore/user-guide.md carries the relevant migration detail.
• docs: update user-guide.md of aicore, destination and object store modules #10 Tightened the broad except Exception: pass in _litellm_patch.py to except ValueError scoped only to the raw_response.json() call. Logic errors in ContentFilteredError construction now surface instead of being swallowed.

Env var prefix renamed from ORCH_FILTER_* to AICORE_FILTER_* for consistency with the new module location.

Class-based API (post-restructure)

The set_filtering() public signature was changed from keyword-driven (set_filtering(hate=..., violence=...)) to class-based (set_filtering(ContentFiltering(input_filtering=InputFiltering(filters=[AzureContentFilter(...)])))). This mirrors the shape used by generative-ai-hub-sdk (gen_ai_hub.orchestration.models.content_filtering.ContentFiltering + InputFiltering + OutputFiltering + AzureContentFilter + LlamaGuard38bFilter) so callers migrating from that SDK keep their call-site structure. Our threshold enum stays Severity.STRICT/LOW/MEDIUM/OFF rather than AzureThreshold.ALLOW_SAFE/ALLOW_SAFE_LOW/ALLOW_SAFE_LOW_MEDIUM/ALLOW_ALL.

AI-generated code disclosure

This contribution was developed with the assistance of Claude (Anthropic). All generated code has been reviewed and validated by the contributor. Tests were written alongside the implementation and verified to pass. Per the SAP AI contribution guideline.

[NicoleMGomes]

if transform_request is public, shouldn't it have the telemetry annotation?

[lenin-ribeiro]

transform_request and transform_response are callbacks on LiteLLM's GenAIHubOrchestrationConfig contract — we override them, but they're invoked by LiteLLM, never by SDK users (the class isn't in __all__). The "public methods get telemetry" rule applies to our entry points (set_filtering, disable_filtering, extract_filter_blocked), all of which are decorated. Adding @record_metrics here would double-instrument
every LiteLLM request — every call already records its own LiteLLM span. Happy to add a counter on the ContentFilteredError raise sites if you'd like an explicit "filter blocked" metric, but a request-level decorator on the callback feels like the wrong place.

[NicoleMGomes]

Couldn't this use secrets resolver?

[NicoleMGomes]

This shouldn't be under core directly

[lenin-ribeiro]

secrets resolver is used for grabbing credentials, not env var from what I saw. we can go through with this change, but doesn't sound right to me

[NicoleMGomes]

This shouldn't be under core directly

[NicoleMGomes]

Let's merge _litellm_patch with this filtering and have all the public interface in one place.

Also please check code guidelines under docs/. Usually we have the client concept

[NicoleGomes1]

PR Review: #174: feat(orchestration): add content filtering and prompt shield module

Author: lenin-ribeiro Branch: feat/orchestration-filtering → main
Verdict: ⚠️ Needs Minor Work
Summary: Solid, well-structured addition of Azure Content Safety + Prompt Shield to aicore; all CI passes, but a few naming/structure details need attention before merge.

---

A: Process & Compliance

#	Criterion	Status	Finding
A1	PR template complete	✅ Pass	All 9 checklist items ticked; description, type, and how-to-test all present.
A2	Conventional Commits	✅ Pass	All 26 commit headlines match type(scope): description. Commit Validation CI job passes. Note: PR title scope is orchestration while the code lives in aicore — cosmetic mismatch but lint still passes.
A3	Issue linked	⚠️ Warning	No Closes #N / Fixes #N. PR body says "N/A — new feature proposed and implemented by the App Foundation agent team." Acceptable for an internally-initiated feature, but if any backlog issue exists it should be linked.
A4	AI-generated code disclosure	✅ Pass	PR body explicitly discloses Claude/Anthropic assistance and links the SAP AI contribution guideline.

---

B: Security & Sensitive Data

#	Criterion	Status	Finding
B1	No sensitive data in code	✅ Pass	No credentials, tokens, internal URLs, or tenant IDs found in diff. The AICORE_FILTER_TEST_SELF_HARM_PROMPT is explicitly left as an env-var placeholder (never committed) per commit message and integration test.
B2	No sensitive data in PR body	✅ Pass	PR body is clean; code examples use generic placeholders.

---

C: Code Quality

#	Criterion	Status	Finding
C1	CI checks passing	✅ Pass	All 11 required jobs pass: Code Quality Checks, Tests & Coverage, Build & Package, Commit Validation, Check Version Bump, Integration Tests, REUSE Compliance, Analyze (python)/CodeQL, license/cla.
C2	Version bump	✅ Pass	pyproject.toml bumped 0.27.1 → 0.28.0; uv.lock regenerated.
C3	Type hints	✅ Pass	All new public functions and class attributes are fully annotated. from __future__ import annotations used throughout. Internal _install(cfg: Any) uses Any with an explanatory comment to avoid a circular import — acceptable.
C4	No hardcoded values	✅ Pass	All severity defaults use Severity.MEDIUM enum; all env-var keys are string constants at call-sites.
C5	Import organization	✅ Pass	No lazy imports without justification; no duplicate requirements files.
C6	Naming conventions	⚠️ Warning	Two issues: (1) Severity enum members are declared in semantic/severity-ascending order (STRICT=0, LOW=2, MEDIUM=4, OFF=6) rather than alphabetical (LOW, MEDIUM, OFF, STRICT) per guidelines (filters.py:37–48). (2) LlamaGuard38bFilter constructor parameters are in ML-categorization order, not alphabetical — hate and self_harm appear near the bottom rather than sorted lexicographically (filters.py:138–155). Neither affects runtime behavior, but both deviate from the stated convention.
C7	No unused code	✅ Pass	No dead imports or unused variables introduced. Stale ty: ignore directives were cleaned up in the last two commits.
C8	No unjustified new dependencies	✅ Pass	No new runtime dependencies added; litellm was already present.
C9	Proto code freshness	➖ N/A	No .proto files changed.

---

D: API & Design

#	Criterion	Status	Finding
D1	API future-proofing	✅ Pass	Class-based ContentFiltering / InputFiltering / OutputFiltering dataclass-like pattern is well-structured and extensible. Severity IntEnum over bare ints. No create_client() needed — filtering is a configuration helper, not a service client.
D2	Public API hygiene	✅ Pass	__all__ in both filtering/__init__.py and aicore/__init__.py expose exactly the documented 13 public symbols. Internal pieces (_litellm_patch, _install, _active_cfg, _ORIGINAL_CONFIG) are correctly prefixed with _. filters.py (no underscore) correctly signals it IS the public surface.
D3	Breaking changes marked	✅ Pass	This is an additive new feature. set_aicore_config() gains a side effect (filtering activation), which is a behavioral change but not a signature break. The user guide's Migration section covers the sap_cloud_sdk.orchestration → sap_cloud_sdk.aicore rename for any in-flight consumers.
D4	Pagination & tenant filtering	➖ N/A	Not a list/query operation.
D5	Telemetry instrumentation	✅ Pass	@record_metrics(Module.AICORE, Operation.AICORE_SET_FILTERING) on set_filtering, AICORE_DISABLE_FILTERING on disable_filtering, AICORE_EXTRACT_FILTER_BLOCKED on extract_filter_blocked. Module.ORCHESTRATION correctly dropped; operation.py updated with the three new operations.

---

E: Tests & Documentation

#	Criterion	Status	Finding
E1	Tests added/updated	✅ Pass	68 unit tests across test_filtering.py, test_filters.py, test_modules.py, test_patch.py + 22 core/env tests. Integration BDD suite (filtering.feature + test_filtering_bdd.py) covers all 4 live scenarios with clean skip when env absent. tests/aicore/unit/test_aicore.py updated to reflect the new public API.
E2	Documentation quality	✅ Pass	aicore/user-guide.md comprehensively updated: overview, quick start, env-var table, programmatic override, multiple providers, disable, error handling, and migration guide.
E3	Module structure compliance	⚠️ Warning	(1) py.typed is missing from src/sap_cloud_sdk/aicore/filtering/ — sub-packages of PEP 561 packages need their own marker file. The parent aicore/ also lacks it (pre-existing gap), but the new filtering/ sub-package doesn't add one either. (2) config.py is absent; ContentFiltering.from_env() lives in filters.py — this is a deliberate post-review consolidation decision (review comment #14) and the result is cleaner, but it deviates from the standard layout. Not blocking given the documented rationale. (3) AICORE_FILTER_TEST_MODEL and AICORE_FILTER_TEST_SELF_HARM_PROMPT are not documented in .env_integration_tests.example — other integration test env vars for the aicore module (AICORE_CLIENT_ID, AICORE_MODEL, etc.) are listed there.

---

⚠️ Non-Blocking Suggestions

• [C6]: Severity enum members should be in alphabetical order per guidelines: LOW, MEDIUM, OFF, STRICT → filters.py:45–48. The current ordering is semantically meaningful (ascending strictness) but breaks the stated convention.
• [C6]: LlamaGuard38bFilter constructor parameters should be alphabetical. Current grouping (violent_crimes, non_violent_crimes, sex_crimes, …, hate, self_harm, …) follows ML taxonomy order, not lexicographic — filters.py:141–154.
• [E3]: Add py.typed to src/sap_cloud_sdk/aicore/filtering/ (empty file). While the parent aicore/ also lacks one, starting the new sub-package correctly is lower-effort here.
• [E3]: Add AICORE_FILTER_TEST_MODEL and AICORE_FILTER_TEST_SELF_HARM_PROMPT to .env_integration_tests.example alongside the existing AICORE_* vars so contributors know what's needed to run the integration suite.
• [_litellm_patch.py:211]: The except Exception: return None in extract_filter_blocked is intentionally broad (any JSON parsing or dict-navigation failure should silently return None). Consider narrowing to except (ValueError, KeyError, TypeError) to document the specific failure modes explicitly, even if the behavior is unchanged.

---

✅ Things Done Well

• Excellent post-review responsiveness: 14+ review comments from @NicoleMGomes all addressed (collapsing to aicore, removing broad except, Severity enum, class-based API, env helpers, telemetry consolidation). Commit messages are unusually detailed and audit-friendly.
• Clean monkeypatch design: _ORIGINAL_CONFIG capture + idempotent _install(None) restore is robust and testable; the test fixture restore_litellm ensures no test leaks.
• Wire-format alignment: Mirroring gen_ai_hub.orchestration.models.content_filtering shapes means callers migrating from generative-ai-hub-sdk keep their call-site structure with minimal diffs.
• Security-conscious integration test design: AICORE_FILTER_TEST_SELF_HARM_PROMPT stays out of the repo entirely — operator-populated from a secret. Good pattern.
• Separation of concerns in core/env.py: Typed env-var readers are cleanly separated from secret_resolver (credential loader), with clear docstring justification.
• ContentFiltering.from_env() returning None on AICORE_FILTER_ENABLED=false threads correctly through set_filtering() → _install(None) → restores original LiteLLM config. The logic is airtight.