Using a local 8-bit 4B model and a monotonic OR rule for 29 ms compliance redaction

The claim

When AI agents query operational logs, configurations, and tickets via Model Context Protocol (MCP) tools, every output requires credential and privacy scrubbing. Under ISO/IEC 42001 (2023) Annex A.7.6 and EU AI Act Articles 10 and 15, secrets and personal data must never reach cloud models or notes. At the same time, engineers need every software version, transaction ID, and file-path pointer intact to diagnose outages (Nissenbaum, 2004).

Standard redaction approaches fail at one of two extremes when applied to database and infrastructure telemetry.

• On our 77-item synthetic benchmark, regex redactors missed compound secrets or broke file paths. Legacy 8-pattern word-boundary (\b) regexes caught only 12 of 49 synthetic secrets (24.5% recall, 37 missed) and 0 of 27 compound secrets (F1 = 0.387). Broadening those regexes naively (v1.0.0) destroyed 8 of 10 (80.0%) legitimate file-path pointers and CLI flags (28.6% overall false-positive rate).
• On that synthetic benchmark, unfiltered Cloud DLP missed secrets, masked technical IDs, and stalled. Default Google Cloud DLP caught only 28.6% (14 of 49) of synthetic secrets and 0 of 27 compound secrets (F1 = 0.346). Worse, it falsely masked 18 of 18 (100.0%) database technical identifiers (64.3% overall false-positive rate). Under its 100-RPM quota, Cloud DLP also stalled for 57 to 59 seconds on 10.3% of calls (80.7x slower than our burst p95).

In production under ISO/IEC 42001 (2023) Annex A.7.6 (report 10), all 4 output paths run our 33-rule deterministic redactor (Layer 3a). Production pairs Layer 3a with regional Cloud DLP in enforce mode and a 32-character technical_context keep-filter. To eliminate the 10.3% Cloud DLP quota stalls, we evaluated a local 8-bit (Q8_0) 4B single-pass decision model (Layer 3b, 0 output tokens).

In this two-stage firewall, Stage 1 (Layer 3a) is the 33-rule prefix-agnostic grammar (14 key rules plus 19 value rules). Stage 2 (Layer 3b) is the local 8-bit 4B single-pass classifier. On a local GPU, kernel latency is 29.0 ms p50 (29.1 ms in our 323-output replay, 331.9 ms sequential HTTPS p50). Both stages combine under a monotonic logical OR law.

$$\text{Redact}(x) = \text{Stage}_1(x) \lor \text{Stage}_2(x) \ge \text{Stage}_1(x).$$

Monotonicity gives compliance auditors a formal floor guarantee. The local 4B model (Stage 2 / Layer 3b) can add redactions on unstructured credentials, spoken passwords ("the keystore password is..."), doubled-quote CSV flags, SQL ALTER ROLE ... PASSWORD statements, or customer PII. That raises recall from 79.6% (39 of 49) to 100.0% (49 of 49) on the 77-item synthetic suite. However, Stage 2 can never unmask a token matched by the deterministic Stage 1 (Layer 3a) grammar (recall >= Stage 1).

Meanwhile, the 4B model's 4-class sensitivity head (technical_identifier, credential_or_secret, customer_pii, customer_schema_or_sql) separates database UUIDs, tablet IDs, and file pointers (vaultPasswordFile=/etc/x) from real secrets. It reached 100.0% 4-class accuracy (77 of 77) and a 0.0% false-positive rate across all 28 technical IDs and file pointers. Across 323 replayed MCP tool outputs (128 cases), the monotonic union achieved 99.4% recall and a 0.42% false-positive rate (F1 = 0.995, 0.0% quota stalls).

The local GPU inference and quantization experiments in this series were built with Carlos Juzarte Rolo. Carlos is a longtime friend, former colleague at Pythian and NetApp Instaclustr, and local-LLM hardware specialist.

Four design rules for local compliance gatekeepers

1. Enforce a monotonic logical OR between deterministic rules and neural disambiguation. Never let a neural model override a deterministic secret match. Under Stage 1 OR Stage 2, the 4B model only inspects spans that survived Stage 1 or ambiguous context hits (recall >= Stage 1).
2. Drop leading \b word boundaries on secret keys and add a file-pointer grammar. The _ character and camelCase prefixes are regex word characters (\w), so \bpassword\b misses 27 of 27 compound keys (ldapbindpasswd=, db_password=, ysqlPassword:). Match secret suffixes without a leading \b while exempting file-path pointers (vaultPasswordFile=/etc/x).
3. Score a 4-class sensitivity distribution in one prefill pass (0 output tokens). Instead of autoregressive text rewriting, read calibrated float32 class probabilities from a single forward pass (Kadavath et al., 2022). Keep a candidate span unmasked only when P(technical_identifier) >= 0.90.
4. Enforce an 8-bit (Q8_0) quantization floor and bypass binary probability clamping. Moving from BF16 to 4-bit quantization (Q4_K_M or MLX 4-bit) flipped 5.4% to 8.7% of borderline safety verdicts. Running at 8-bit (Q8_0, 4.48 GB VRAM) cut boundary flips to 1.4%.

Why regexes and cloud DLP APIs fail on operational telemetry

Auditing egress across 53 MCP tools and 4 output paths exposed three failure modes in standard redactors (Basak et al., 2023; Lukas et al., 2023).

1. The \b word-boundary failure on 27 compound secrets

Legacy secret scrubbers rely on 8 canonical word-boundary patterns such as (?i)\b(password|passwd|pwd|secret|token|api[_-]?key)\b. In regular expression engines (Cox, 2007), ASCII letters, digits, and _ are all word characters (\w). A word boundary \b matches only at a transition between \w and \W.

When an application prefixes a key via camelCase or snake_case (ldapbindpasswd=, db_password=, ysqlPassword:, client_secret=, aws_secret_access_key=), the character before password or secret is already \w. The leading \b never matches. In our 33-pattern audit corpus, the legacy \b redactor missed 27 of 33 (81.8%) compound keys, catching only 6 of 33 (18.2%). On the 27 synthetic compound secrets in our 77-item benchmark, legacy \b recall was 0.0% (0 of 27). An offline regex-design scan of historical process-table snapshots (pidstat) before indexing also cataloged 113,000 command-line password= or secret= arguments (113,499 exact) and 14,911 Java -pw VALUE flags.

2. The 80.0% file-pointer false-positive trap in naive broad regexes

Removing \b naively (v1.0.0 draft ruleset) created the opposite problem. In an offline dry run over configuration tables, naive broad regexes flagged 1,818 database rows, of which 1,277 (70.2%) were false positives. Across 11,260 configuration rows containing keyInfo.privateKey or kubeConfig, 11,248 rows (99.89%) stored filesystem paths (such as /opt/app/keys/synth.pem) rather than secret key material.

On our 10-item file-pointer and CLI-flag benchmark subset, the naive broad regex falsely redacted 8 of 10 (80.0%) technical items. It destroyed privateKey paths, KUBECONFIG paths, vaultPasswordFile paths, --ysql_password_file flags, password_encryption=scram-sha-256, psql -p 5433 port flags, PWD=/home/app working directories, and Java stack frames (PasswordPolicyService.java:85).

Our 33-rule Stage 1 (Layer 3a) grammar (v1.3.0, 14 key rules and 19 value rules) solved both regex defects deterministically. It removed the leading \b boundary, added an explicit file-pointer exemption grammar, and parsed CLI flags by binary context (masking -pw VALUE and mysql -pSecret while preserving psql -p 5433). That lifted Stage 1 precision to 100.0% (0 of 28 false positives).

3. Why default Cloud DLP masked 18 of 18 database IDs

On the 77-item suite, default Google Cloud DLP missed all 27 compound secrets (0.0% recall) and 6 of 8 bundle secrets (25.0% recall). Arbitrary alphanumeric strings inside ldapbindpasswd= or PLAY_SESSION= do not match statistical named-entity patterns for natural-language PII.

At the same time, Cloud DLP falsely masked 18 of 18 (100.0%) database technical identifiers (64.3% overall FPR across the 28 negative items). Its PHONE_NUMBER detector masked backup archive names (backup_26-10-11-07-00.tgz) and 8-digit RPC call IDs (13008957, 44819203). Its ENCRYPTION_KEY detector masked replication stream UUIDs (<uuid>_xcluster), Raft tablet IDs, and RocksDB SST table hex IDs. Its CREDIT_CARD_NUMBER detector masked 19-digit hybrid timestamps and transaction priorities, while its IP_ADDRESS detector masked software version strings (v2.20.1).

As reported in report 10, wrapping Cloud DLP in a deterministic 32-character technical_context keep-filter rescued all 18 of 18 database IDs (0.0% false masking). However, a keep-filter around a remote cloud API still suffers 100-RPM quota stalls (57 to 59 s on 10.3% of calls) and external network egress.

Six-arm benchmark across 77 adversarial items and 323 tool outputs

We evaluated six redaction arms on our pre-registered 77-item adversarial suite (49 positive secret, PII, and SQL items; 28 negative technical items). We also replayed the two-stage firewall across 323 production MCP tool outputs (128 cases).

Comparing Arm 4 (Stage 1 deterministic grammar alone) with Arm 6 (Stage 1 OR Stage 2 monotonic union) shows what each stage contributes. Stage 1 catches 39 of 49 positive items (79.6% recall) in linear time with 0 false positives (F1 = 0.886). However, a pure regular-expression grammar cannot safely match bare keys like passwords= or auth= without risking false positives elsewhere. Nor can it parse unstructured phone numbers, SQL query literals, or state-conditioned cross-tenant table names.

Stage 2 (8-bit 4B, Layer 3b) rescues all 10 items missed by Stage 1, lifting recall from 39 of 49 (79.6%) to 49 of 49 (100.0%).

• 3 compound and SQL DDL secrets in the compound-secret subset (24/27 to 27/27). These are passwords=Zq7xS3cr3tV4lue9, secret_key=Zq7xS3cr3tV4lue9, and SQL ALTER ROLE app WITH LOGIN PASSWORD 'Zq7xS3cr3tV4lue9'.
• 1 bare canonical key in the canonical-secret subset (7/8 to 8/8). This item is auth=Zq7xS3cr3tV4lue9.
• 1 CSV-escaped bundle flag in the bundle-configuration subset (7/8 to 8/8). This item is --ysql_hba_conf_csv with doubled quotes (ldapbindpasswd=""Zq7xS3cr3tV4lue9"").
• 5 customer PII and SQL items in the PII and SQL subset (1/6 to 6/6). These cover a bridge phone (+1-415-891-7742), SSN (901-44-8821), IBAN (GB29NWBK60161331926819), card PAN (4532015112830366), and cross-tenant table (tenant_bank_a_settlements.ach_transfers).

Why keep Stage 1 (Layer 3a) at all if standalone Stage 2 (Arm 5) also scored 49 of 49 on the 77-item suite? First, Stage 1 scrubs 79.6% of secrets in microseconds before any neural call runs. Second, under ISO/IEC 42001 (2023) Annex A.7.6, a neural model alone lacks a deterministic floor against prompt injection (Abdelnabi et al., 2026; Bagdasarian et al., 2024). Under the monotonic OR rule, even if an untrusted log line spoofs the 4B classifier, nothing matched by the 33 rules is unmasked.

How the 4-class single-pass head resolves context in 29 ms

The Stage 2 (Layer 3b) 4B decision model (report 07) runs 24 causal Gated DeltaNet layers followed by 8 bidirectional full-attention layers (32 layers total). Its top 8 layers attend bidirectionally across the telemetry excerpt, the session metadata (active_customer_org), and the candidate span. One prefill pass (0 output tokens) emits a calibrated float32 probability distribution across four sensitivity classes.

1. Technical identifier (technical_identifier). Software versions, backup filenames, RPC call IDs, replication UUIDs, hybrid timestamps, port flags, and filesystem path pointers.
2. Credential or secret (credential_or_secret). Passwords, bind secrets, bearer tokens, session cookies, cloud access keys, and spoken or SQL-embedded credentials.
3. Customer PII (customer_pii). Natural-person emails, phone numbers, tax identifiers, and payment card numbers.
4. Customer schema or SQL (customer_schema_or_sql). Customer-specific SQL literals (such as wire IBANs) and cross-tenant schema or table names belonging to another organization.

Across all 77 benchmark items, the 4-class head predicted the exact ground-truth sensitivity class on 77 of 77 items (100.0% 4-class accuracy).

On the 43 credential items, P(credential_or_secret) ranged from 0.950 to 0.979. On customer PII and cross-tenant SQL items, the matching sensitive class scored 0.900 to 0.980. Conversely, on version strings, RPC call IDs, backup archives, and file pointers (vaultPasswordFile=/etc/x), P(technical_identifier) scored 0.960 to 0.991 (0.962 to 0.980 in the examples above).

Replayed across 323 production MCP tool outputs (128 incidents), the monotonic firewall reached 99.4% recall at a 0.42% false-positive rate (99.58% ID preservation, F1 = 0.995). Unfiltered Cloud DLP on the same 323 outputs masked 100.0% of the 18 database ID classes (0.0% preservation, 0.720 F1). It also stalled on 10.3% of calls (89.7% zero-stall availability vs 100.0% on the local firewall).

Latency, tail stalls, and why 8-bit quantization is the compliance floor

A privacy firewall that sits in front of 53 interactive MCP tools must operate within interactive tool budgets. In our benchmark, every single-pass 4B call verified output_tokens = 0 (100% of calls).

• Local GPU kernel latency (29.0 to 29.1 ms). On a local GPU with 8-bit (Q8_0) weights, median prefill latency is 29.0 ms (29.1 ms on 323 outputs) with 0 cloud bytes.
• Remote HTTPS latency (331.9 ms sequential p50, 730.7 ms 6-way burst p95). Over remote HTTPS keep-alive (77 calls), sequential latency was 331.9 ms p50 (323.8 ms min, 404.2 ms p95). Under 6-way concurrent agent fan-out, latency was 489.7 ms p50 and 730.7 ms p95 (1,048.8 ms p99, 1,081.2 ms max).
• Eliminating the 57 to 59 second Cloud DLP quota stalls (80.7x tail speedup). Under fan-out, Cloud DLP hit its 100-RPM quota (ResourceExhausted) and stalled for 57,000 to 59,000 ms (57 to 59 s) on 10.3% of calls. At 6-way HTTPS concurrency (730.7 ms p95), our 4B stage is 80.7x faster than 59-second stalls (145.9x sequential p95, >2,000x local GPU).

Qualifying the local 4B model for ISO/IEC 42001 (2023) compliance gates also required enforcing two numerical stability floors established in report 07 and report 11.

1. Why 4-bit quantization is banned (8-bit Q8_0 floor at 4.48 GB VRAM). Quantizing the 4B model from BF16 (16-bit) to 4-bit moved net benchmark accuracy by less than 0.3 percentage points. Yet on borderline spans near the decision boundary, 4-bit quantization flipped 5.4% (GGUF Q4_K_M) to 8.7% (MLX 4-bit) of safety verdicts relative to BF16. Running at 8-bit (GGUF Q8_0, 4.48 GB VRAM) cut boundary flips to 1.4% (98.6% BF16 agreement), well below our 2.0% compliance ceiling.
2. Why multi-class choice replaces clamped binary scoring (0 flips in 324 checks). On hosted endpoints, a default 0.801 floor clamps binary probabilities in (0.199, 0.801) outward to the boundaries. Near-threshold items then suffer a 2.47% (8 of 324) repeat-run flip rate. Querying the unclamped multi-class choice head produced 0 verdict flips across 324 pairwise comparisons (10 repeated runs, 0.0% flip rate).

Verification gates and methodology checks

• Monotonicity invariant audit (Stage 1 OR Stage 2 >= Stage 1). Verified across all 77 adversarial items and 323 replayed outputs that Stage 2 (Layer 3b) only adds redactions (+10 rescued items) and never unmasks Stage 1 (Layer 3a).
• Single-pass zero-generation check (output_tokens = 0). Verified in our benchmark ledger that 100% of Stage 2 calls returned output_tokens = 0, preventing generative hallucination or text corruption.
• File-pointer dry-run audit (11,248 of 11,260 rows). Verified that our file-pointer path grammar eliminated the 70.2% (1,277 of 1,818 rows) naive-regex false-positive rate on filesystem paths (0 of 10 benchmark pointer FPs).
• Quantization and repeat-run stability gate (K=10 repeats, 324 comparisons). Verified that 8-bit Q8_0 (1.4% boundary flips vs BF16) and unclamped choice (0 of 324 repeat-run flips) meet our pre-registered <= 2.0% flip ceiling.

Redaction antipatterns rejected by our gates

1. Leading word-boundary \b anchors on secret key regexes (missed 27 of 27 compound camelCase and prefixed secrets on the 77-item suite, and 27 of 33 in our audit corpus).
2. Naive broad substring regexes without a file-pointer grammar (caused a 70.2% false-positive rate across 1,818 database rows and destroyed 8 of 10 file pointers and CLI flags).
3. Unfiltered Google Cloud DLP on operational logs (missed 27 of 27 compound secrets, falsely masked 18 of 18 database IDs, and stalled 57 to 59 s on 10.3% of calls).
4. Allowing a neural model to override deterministic secret rules (rejected in favor of the monotonic logical OR rule Stage 1 OR Stage 2 >= Stage 1).
5. Deploying 4-bit quantized models (Q4_K_M or MLX 4-bit) as compliance gatekeepers (rejected after 4-bit weights flipped 5.4% to 8.7% of borderline safety verdicts).

Pressure tests

Every finding was checked against the six standing fronts in Methods.

What this does not show

First, our production stack runs Stage 1 (Layer 3a, 33 rules) plus regional Cloud DLP (enforce mode) and our 32-character technical_context keep-filter (report 10). The 6-arm benchmark (77 synthetic items) and 323-output replay evaluate our local 8-bit 4B Stage 2 (Layer 3b) gatekeeper before replacing the Cloud DLP hop.

Second, the 0.42% false-positive rate on the 323 replayed production MCP tool outputs reflects our conservative default-deny threshold (keep unmasked only when P(technical_identifier) >= 0.90). When an unfamiliar custom hostname contains a substring like pwd (ldap.pwd-site...:636, P(technical_identifier) = 0.484), our < 0.90 gate errs on the side of masking or escalating.

Third, our two-stage firewall operates on UTF-8 text payloads returned by MCP tools, SQL queries, and log parsers. Binary artifacts (such as .hprof heap dumps or core dumps) are blocked at ingestion by our Annex A.4.4 archive guard (report 10) rather than line-scrubbed.

Finally, the monotonic OR rule (Stage 1 OR Stage 2) guarantees that an adversarial log injection can never unmask a deterministic Stage 1 match. However, novel unstructured secrets that evade Stage 1 still rely on Stage 2 classification. Pairing structural JSON field isolation with the 0.90 keep-probability floor bounds that residual surface.

Reproduce

All metrics in this report and its four figures are stored in numbers.json, with SHA-256 integrity digests recorded in MANIFEST.md.

References

• Abdelnabi, S., Gomaa, A., Bagdasarian, E., Kristensson, P. O., & Shokri, R. (2026). Firewalls to Secure Dynamic LLM Agentic Networks. Transactions on Machine Learning Research (TMLR). arXiv.
• Bagdasarian, E., Yi, R., Ghalebikesabi, S., Kairouz, P., Gruteser, M., Oh, S., et al. (2024). AirGapAgent: Protecting Privacy-Conscious Conversational Agents. Proceedings of ACM CCS 2024. DOI. arXiv.
• Basak, S. K., Cox, J., Reaves, B., & Williams, L. (2023). A Comparative Study of Software Secrets Reporting by Secret Detection Tools. Proceedings of ACM/IEEE ESEM 2023. DOI.
• Cox, R. (2007). Regular Expression Matching Can Be Simple And Fast. Technical Note.
• European Parliament & Council of the European Union. (2024). Regulation (EU) 2024/1689 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). Official Journal of the European Union.
• ISO/IEC. (2023). ISO/IEC 42001 (2023): Information Technology - Artificial Intelligence - Management System. International Organization for Standardization.
• Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., et al. (2022). Language Models (Mostly) Know What They Know. arXiv preprint. arXiv.
• Lukas, N., Salem, A., Sim, R., Tople, S., Wutschitz, L., & Zanella-Beguelin, S. (2023). Analyzing Leakage of Personally Identifiable Information in Language Models. Proceedings of IEEE Symposium on Security and Privacy (S&P 2023). DOI.
• Nissenbaum, H. (2004). Privacy as Contextual Integrity. Washington Law Review.