Local synthetic evaluation · 2026-10-04 · shadow

Russian PII & secrets,
measured.

Cloud.ru guardrails-llm-filter runs after mandatory Presidio at the native LiteLLM gateway. The pilot records aggregate detections. Enforcement remains a separate validation step.

Where each configured control helps

The 64-case suite contains 50 sensitive values and 14 benign probes, including difficult OCR, lowercase, spoken and Unicode forms. Secrets and addresses are outside this Presidio policy. Aggregate totals do not establish a general tool ranking or production recall. Partial masking is a miss; annotations include formatting characters.

Observed limits

Cloud.ru missed lowercase and unusual names, some address/OCR forms, Cyrillic or spoken email, compact/spoken/OCR phones, an invalid-checksum SNILS, an OCR passport and a Russian password label. Presidio protected more contact probes: 7/10 versus 5/10.

Local performance and fault behavior

Warm sequential HTTP p95: Cloud.ru 6.14 ms, configured Presidio 16.46 ms, residual scan 6.92 ms. This excludes model inference and is not a production SLA.

Seven direct callback checks cover input/output and private audit, embedding/rerank, scanner outage, capacity, input limits and unsigned identity denial. Presidio stays mandatory during scanner failure. Residual detections do not change the returned content in shadow mode.

Reproducible evidence

Download the full aggregate report · Upstream scanner

No customer data, hosted model calls or real user identities were used. The JSON includes miss IDs, runtime image IDs and source fingerprints; no prompts or original values.

Suite SHA-256
2c9f8e6428f44f0c479aa518e409117b09798435b1bb6d39289437119af2a7b8