All-in-one AI/LLM firewall · low latency

All-in-one AI/LLM firewall
intelligent, self-learning, low latency

InfernoFirewall combines ML-driven fast decisions, client protection, DLP, accounting, tool-call safety and policy guardrails in one gateway. It learns from traffic and incidents, then tightens only what it can prove — keeping the fast path under 10 ms.

p50 < 3 msfast ML decision path, full path measured
p99 < 10 mslow-latency enforcement; tier-2 judge only for high-impact calls
Client protection + DLPclients, accounts, tool calls, prompts, responses, usage accounting
Self-learningtraffic and incidents feed candidate rules that only tighten the ceiling

The core differentiator: a hard, provable ceiling

Most AI guardrails are a model judging a model — the judge can be fooled, and you never know which requests it waved through. InfernoFirewall inverts it: the ML layer emits facts, and a Cedar policy engine decides. Because ML facts only appear inside forbid conditions, the allowed set can only shrink. The bound you can state before deployment is the product.

Allowed(r) = Permitstatic(r) ∧ ¬Forbidstatic(r) ∧ ¬ForbidML(r)
// ML can shrink the set. It can never widen it.
@forbid @verdict("step_up")
forbid(principal, action == Action::"send_email", resource)
when { context.injection_score >= 0.8 };

// Before a bundle ships: invariants proven over every request the
// schema allows (cvc5), and a semantic diff with real counterexamples.
// A loosening needs a named human. Tightenings can ship without one.

Formally analyzable policy

Semantic diff and invariants. Prove statements like "no prod principal can ever call shell_exec" or "nothing labeled secret reaches an external sink" — the SMT solver answers before the policy ships, with counterexample requests when it doesn't hold.

Tool calls are the attack surface

Agent-first, not text moderation. The real risk is an agent taking a harmful action because untrusted content steered it. Every tool call is an untrusted proposal until policy permits it.

Sub-10 ms fast path

Three tiers. Compiled patterns + schema + Cedar in under 1 ms on every event; a calibrated compact classifier < 6 ms on CPU; an LLM judge only when the classifier is uncertain and the action is high-impact.

Tamper-evident, replayable audit

Every decision is a hash-chained record — inputs, bundle hash, rules fired, scores, approver. Checkpoints are signed with Ed25519, and a verifier CLI exits 1 on any edit or gap. Replay reproduces every verdict; what-if runs a candidate policy over history.

Taint that only grows

Provenance per session. Anything a tool returned is untrusted for the rest of the session, and it flows to summaries, memory and child agents. Policy decides what taint means; it clears only through an audited declassification step.

Approvals bound to exact arguments

Single use, re-decided on release. An approval is tied to the canonical arguments hash — approving $100 never releases $1000. Approver authority is itself a policy, and no approval can override a hard deny.

Tool-description pinning

Against MCP rug pulls. Descriptions and schemas are hashed and pinned at approval. Change them upstream and the tool disappears until re-approval; tool text that contains injection instructions is blocked at sync.

Fail closed

By construction. Deadline exceeded, stage error, panic, unparseable JSON, lost session state: tool calls are always denied. Only plain chat text can opt into failing open.

Outbound data controls

Providers are sinks. The whole request — history included — is scanned, and each backend is decided by its trust class: self-hosted, private or public. Secrets never leave; personal data stays inside commercial APIs.

Learns, but only tightens

Co-evolution with a guard. Blocks and red-team findings are mined into candidate rules; anything the analyzer proves to be a tightening can auto-ship, anything looser requires a named human approver in the audit trail.

Shadow mode

Adopt without disruption. Run the firewall over real traffic, record what it would have blocked or held, and enforce when the false-positive budget is provably acceptable.

Honest evaluation

Four configurations, always reported together: no firewall, static policy only, classifier only, combined — attack success, benign task success, approval burden and latency. We publish what is measured and label what is synthetic.

One tool call, end to end

A single in-process pipeline handles every event type and ends in one decision. The control plane is never on the request path.

  1. Hold the call. Text streams through; tool-call deltas are buffered until the arguments JSON is complete — nothing executes before it is decided.
  2. Tier 0 (µs). Arguments validated against the tool's schema; ~35 pattern rules scan for secrets, PII and injection strings — including encoded views (base64, homoglyphs, zero-width characters).
  3. Tier 1 (< 6 ms). A compact calibrated classifier scores injection, exfiltration and harmful intent per window; thresholds come from Learn-then-Test, not guesses.
  4. Provenance. The session graph adds where the inputs came from: taint, data classes, sequence signals like secret_then_sink.
  5. Cedar decides. Allow, deny, redact (arguments rewritten, then re-authorized), or step-up — held as a durable approval bound to the exact call.
  6. Results are treated the same. Every tool result is scanned and recorded before it re-enters the model's context; the session becomes untrusted.
  7. The record stands. Consequential calls are committed to the hash-chained audit log before they run — a crash can never leave an executed action with no record.

Results, stated honestly

The harness reports four configurations side by side so a drop in attacks can't be a blanket denial in disguise. Numbers below are from the last run — synthetic scenarios on a scripted model, published with the method so they can be checked. No public suite has been run with a real model yet; latency figures are design targets pending reference hardware.

Metricno firewallstatic policy onlyclassifier onlycombined
Attack success (44 scripted attacks)100%0%0%0%
Benign tasks completed (48 sessions)100%64.6%68.8%54.2%
Step-ups per 100 sessions060.907.6
Added latency p99, gateway stages1.0 ms≤ 2.5 ms≤ 10 ms≤ 5.0 ms
Adaptive red-team: variants evaded32/320/329/32 after 5 rounds0/32

The same honesty as the design doc: the hard guarantees come from the policy and flow layers; the ML layer improves precision with statistical, not absolute, bounds — and it can only tighten.

Architecture — one enforcement core, several modes

Architecture diagram: deployment modes (AI gateway, MCP gateway, SDK middleware, network modes) feed one canonical-event pipeline — adapters, tier 0, tier 1, session graph, Cedar + decision wrapper — with the control plane off the request path, Postgres session state, the audit chain, and async export sinks

What each mode sees and stops

Every mode shares the same pipeline, policy engine and audit chain, so a guarantee proven once applies to every mode. Each mode states exactly what it sees and what it can stop.

ModeSeesCan stopStatus
AI gatewayAll prompts, responses, proposed tool callsRequests; responses; tool calls by rewriting the responseBuilt
MCP gatewayTool calls and results for routed serversExecution of routed tools; pinned, scanned tool listsBuilt
Web consoleLive decisions, sessions, policy, approvalsCedar-role-gated changes, audited before they happenBuilt
SDK middlewareIn-process tool callsIn-process tool callsPlanned
Provider-compatible proxySame as AI gateway, for catalog hostsSame; other endpoints on those hosts refusedBuilt
Network modesAI-destination flows (DNS/SNI), TLS-inspected when allowedWhole destinations; tool calls in inspected responsesLater phases
Built: OpenAI-compatible gateway (streaming too) Built: Cedar + decision wrapper, strict schema, fail-closed Built: taint, flow rules, budgets, child agents Built: durable single-use approvals Built: hash-chained audit, replay, per-tenant export sinks Built: console with live feed, shadow mode, usage attribution Built: provider-compatible CONNECT proxy for catalog hosts Built: calibration (Learn-then-Test) tooling — production model pending Not yet: independent review, reference-hardware latency, first partner

Run your agents through the firewall — in shadow mode first

Self-hosted today: a single Rust binary and Postgres, Docker Compose for the quick start. Design partners run real agents in shadow mode before enforcing anything. Deny by default, prove the ceiling, then open the gates with the diff.