Securing the Multi-Tenant AI Frontier: Inside Keybound and the Audit of LLM Prompt Cache Isolation

Executive Overview

As Large Language Models (LLMs) transition from experimental novelties to core enterprise infrastructure, the architecture supporting them must evolve accordingly. To maximize efficiency, cut operational costs, and reduce latency, modern enterprise deployments frequently rely on multi-tenant LLM relays and gateways. These intermediaries allow multiple distinct organizations, business units, or internal teams (tenants) to route their API requests through a shared pool of upstream credentials or infrastructure.

However, this pooling mechanism introduces a severe, often overlooked vulnerability class: prompt cache contamination and collapse.

When a multi-tenant relay utilizes a single upstream credential to serve multiple isolated clients, the underlying caching layers—designed to optimize prompt evaluation by storing prefix states—can suffer from namespace collapse. Put simply, the boundary separating tenant data dissolves. Tenant B can inadvertently read prompt cache states written by Tenant A, leading to data leaks, regulatory non-compliance, and severe security breaches without either party ever knowing.

Enter keybound, a specialized open-source auditing tool designed to rigorously verify whether multi-tenant LLM relays comply with the defense contract laid out in academic research paper arXiv:2608.17485 (KeyPooling). Developed by security researcher Pedro Sordo Martínez (known online as Fenix / magopredator), keybound acts as an automated governance and verification engine. It subjects mock and live LLM gateways to rigorous formal cell audits to determine whether tenant isolation is strictly enforced at every cache lookup and write cycle.

This article provides an exhaustive look into the mechanics of prompt cache leaks in multi-tenant relays, the core tenets of the KeyPooling defense contract, how keybound evaluates compliance, and what this development means for the future of enterprise AI security.

Detailed Chronology: The Genesis of Keybound and Multi-Tenant Cache Auditing

The realization that multi-tenant LLM proxies represent a blind spot in enterprise security did not happen overnight. It is the culmination of architectural scaling challenges and academic research into inference optimization vulnerabilities.

Phase 1: The Rise of Multi-Tenant LLM Relays and Upstream Pooling

To manage high volumes of traffic and bypass strict rate limits imposed by providers like OpenAI, Anthropic, or custom internal model endpoints, engineering teams widely adopted open-source and proprietary proxy relays (such as LiteLLM, NewAPI, and custom enterprise gateways). These relays often pool upstream API keys. When a request hits the relay, the proxy strips the client’s individual tenant token, substitutes a master corporate API key, and forwards the payload upstream.

While this approach simplifies credential management, it wreaks havoc on stateful features like prompt caching. Modern LLM inference engines (such as vLLM, TGI, and proprietary cloud APIs) cache prompt tokens to accelerate subsequent generations. Caching decisions are typically made based on the exact token prefix and the authorization context or namespace provided in the request headers. If a relay collapses these contexts into a monolithic upstream connection, the cache engine loses sight of individual tenant boundaries.

Phase 2: The KeyPooling Blueprint (arXiv:2608.17485)

Recognizing this systemic architectural flaw, security researchers published academic literature—most notably arXiv:2608.17485 (KeyPooling)—establishing formal security requirements for multi-tenant LLM intermediaries. The paper introduced a strict "defense contract" designed to govern how identities, tokens, and cache namespaces must interact.

The core mandate of the KeyPooling defense contract is unequivocal:

“A namespace derived from authenticated identity must survive every final cache lookup and write.”

Under this model, even if requests are multiplexed over a shared upstream credential, the downstream tenant’s unique cryptographic or authenticated identity must inform the cache namespace. If a relay fails to inject, preserve, or isolate this namespace during either a lookup (GET) or a write (PUT), the defense contract is breached.

Phase 3: The Birth of keybound

Despite the clarity of the KeyPooling paper, engineering teams lacked an automated, deterministic way to verify whether their chosen relays actually honored this contract. Manual penetration testing of LLM caching behavior is notoriously difficult due to the probabilistic nature of generation and the black-box nature of commercial inference engines.

To bridge this gap, Fenix developed keybound. Conceived as a deterministic auditing engine, keybound automates the execution of formal E2 (End-to-End) test cells against LLM gateways. Rather than relying on fragile end-to-end integration tests over live networks, keybound implements a rigorous synthetic test harness that evaluates cache isolation behavior under adversarial permutations—ensuring that prompt cache segregation is mathematically verified before code ever reaches production.

Supporting Context & Technical Metrics: How keybound Works

To understand the sophistication of keybound, one must examine its internal evaluation engine, its handling of state, and its strict privacy guarantees.

The Mechanics of the Audit

When a user executes an audit via the command-line interface, keybound does not simply check configuration files; it simulates complex, multi-actor interaction sequences designed to trick the relay into leaking state.

keybound audit --fixture collapse
# fixture: collapse | verdict: FAIL | defense_contract: not_satisfied
# cause: 1 domain(s); cross_con_leak=['A1-cold-prime', 'A2-owner-hot']

The tool operates on a set of foundational principles:

1. Identity-Based Verification: Veredicts are determined purely by real identity, never by mere cell labels. A data leak is formally defined as any cache lookup resulting in cached_tokens > 0 where the writer of that cache block (hit_writer) belongs to a different tenant than the reader.
2. Order Invariance: Adversarial permutations—such as running a cross-tenant read cell (cross) before the legitimate owner’s cold or hot write cells (cold/owner)—do not fool the auditor. The state machine tracks provenance independently of execution sequence.
3. Strict Zero-Knowledge Reporting: Enterprise security auditing tools must be careful not to exacerbate the vulnerabilities they uncover. keybound strictly ensures that generated reports never include sensitive tenant secrets, raw prompts, or the received Authorization headers. Only non-sensitive identifiers (Tenant A/B, prompt designators P/R, and effective isolation domains) are logged.

Three-Part Governance and Bug Hunting

The development of keybound followed a rigorous three-part governance protocol:

1. Implementation: Core auditing logic written in Python.
2. Independent Audit: Secondary validation executed in a clean clone environment.
3. Merge Approval: Strict gating before code integration.

This rigorous methodology successfully intercepted and neutralized three distinct architectural bugs during development, ensuring that the auditor itself remains completely resilient against false positives and state-handling blind spots.

Official Statements and Architectural Philosophy

While keybound currently operates against a synthetic, deterministic mock gateway to maintain lightning-fast CI/CD execution suites, its architectural design anticipates real-world deployment complexity.

In the official documentation and design notes (RESEARCH.md and KNOWN_ISSUES.md), the maintainer emphasizes that the tool is built for extensibility:

"The fast test suite validates the logic of the auditor itself, rather than re-confirming upstream proxy vulnerabilities on every CI run. Real-world vulnerabilities identified in popular relays (such as unpatched routing in multi-tenant LiteLLM or NewAPI configurations) are captured via dedicated adapters scheduled in the v0.3 roadmap."

The choice of the AGPL-3.0-or-later license further underscores the project’s commitment to open-source governance. By utilizing a strong copyleft license, the author ensures that any enterprise modifying or embedding keybound within proprietary security scanning suites must contribute their improvements back to the community, fostering a collaborative defense against emerging LLM infrastructure threats.

Future Outlook: The Road Ahead for LLM Cache Security

As organizations scale their generative AI deployments, efficiency will remain paramount. Prompt caching is no longer an optional optimization; it is a financial necessity that slashes enterprise API bills by up to 50% and dramatically reduces time-to-first-token (TTFT). However, efficiency must never outpace security.

The release of keybound marks a crucial turning point in how the AI engineering community approaches infrastructure auditing. Moving forward, we can expect to see several key trends:

1. Shift-Left AI Security: Tools like keybound will become standard fixtures in CI/CD pipelines for organizations deploying custom LLM proxies and gateways. Auditing proxy configuration for cache isolation will be treated with the same gravity as static application security testing (SAST) for SQL injection.
2. Upstream Gateway Hardening: As frameworks like LiteLLM, Portkey, and Langfuse integrate tighter multi-tenant boundary controls, automated auditors will serve as the definitive benchmark for compliance against standards like KeyPooling.
3. Expansion of Protocol Adapters: With roadmap milestones targeting adapters for NewAPI and other enterprise routing fabrics, keybound is positioned to evolve from a specialized academic verification script into an industry-standard security utility.

Getting Started with keybound

For security engineers, DevOps professionals, and platform architects looking to verify their multi-tenant LLM infrastructure today, the tool is fully open-source and ready for local evaluation:

git clone https://github.com/amurlaniakea/keybound
cd keybound
python -m venv .venv && source .venv/bin/activate
pip install -e .

# Run the collapse fixture (expected failure due to namespace collision)
keybound audit --fixture collapse

# Run the isolate fixture (expected success for properly segmented proxies)
keybound audit --fixture isolate

As the frontier of artificial intelligence expands into hyper-optimized, multi-tenant environments, tools like keybound provide the essential guardrails needed to ensure that speed and efficiency do not come at the cost of data privacy.