Multi-Tenant AI: Running Isolated Agents Across Hundreds of Businesses

The Challenge: AI Makes Multi-Tenancy Harder

Multi-tenancy in a CRM is well-understood: filter every query by company_id, add some RLS policies, move on. But multi-tenant AI adds three problems traditional SaaS doesn't have:

• AI agents have memory. If Sarah (customer service) remembers Company A's preferences, that memory must never appear in Company B's conversations. Memory isolation is a new category of tenant boundary.
• AI agents have credentials. Each company can bring their own OpenAI or Anthropic API keys. If SmartRouter accidentally routes Company A's request through Company B's key, you've cross-contaminated billing and potentially violated data agreements.
• AI agents have knowledge. Each company has a custom knowledge base — services, pricing, FAQ, brand voice. Injecting the wrong KB into a conversation isn't just a bug. It's a business owner seeing a competitor's pricing in their own AI assistant.

At Solid#, we chose the hardest multi-tenancy model for the best economics: shared database, shared schema, shared backend. One PostgreSQL database serves every company. One FastAPI server handles every request. The isolation is entirely in the application layer and database policies.

The Architecture: 7 Isolation Layers

Layer 1: JWT Authentication

Every authenticated request carries a JWT with the user's company_id. Middleware extracts this before any controller logic runs. The company ID isn't a parameter the user provides — it's derived from their authenticated session. You can't request another company's data because you can't forge the JWT.

Layer 2: Application-Layer Query Filtering

Every database query includes WHERE company_id = X. This is enforced through FastAPI dependency injection — controllers receive the company from middleware, not from request parameters. There's no code path where a query runs without a company filter.

This is the primary enforcement layer. It's explicit, auditable, and testable. Every developer can see the filter in the code.

Layer 3: Row-Level Security (Defense-in-Depth)

298 tables have PostgreSQL RLS policies that enforce company_id filtering at the database level. Even if application-layer filtering has a bug, the database rejects cross-tenant queries. The session variable app.current_company_id is set at request time from the JWT.

This is defense-in-depth — if the application layer fails, the database catches it. RLS is the safety net, not the primary mechanism.

Layer 4: Agent Isolation

Every agent, conversation, and memory is scoped to a company:

• Agents: Each company gets their own agent instances with company-specific settings
• Conversations: Every conversation has a company_id + agent_id pair. Unique conversation IDs prevent cross-tenant access by ID collision.
• Memory: Persistent memory is keyed by company_id in both Redis and PostgreSQL. No wildcard queries.
• KB Scope: Agents can be restricted to specific knowledge base categories per company, so a customer service agent only sees FAQ content while a sales agent sees pricing.

Layer 5: Credential Isolation

Companies that bring their own LLM API keys get Fernet-encrypted storage (AES-128). Each company's keys are stored in the llm_providers table with their company_id. SmartRouter validates that an agent's company matches the provider's company before making any API call. A routing bug can't accidentally use another company's credentials.

Layer 6: Knowledge Base Isolation

Each company gets a knowledge base cloned from one of 52 industry templates at provisioning. A plumber gets plumbing terminology, service menus, and FAQ. A dental practice gets dental content. The KB is then customizable per company, but the starting point is industry-appropriate.

The 4-layer KB architecture ensures context never leaks:

1. GPT Contexts — per-company personality, identity, and brand voice
2. Company KB — per-company services, pricing, FAQ (from industry template)
3. Master KB — platform-wide fallback content (shared, read-only)
4. Agent KB Scope — per-agent access control within a company's KB

Layer 7: Budget Isolation

Every company has its own AI budget tracked in Redis, enforced by CognitiveLimiter. Company A consuming their entire daily budget has zero impact on Company B. Budget keys are namespaced: daily_spend:{company_id}:{date}. No shared counters.

What Provisioning Creates

When a new company signs up, the provisioning system creates ~700 rows across dozens of tables in 2-5 seconds. Every row has the new company_id:

After provisioning, the company is immediately accessible at company-slug.solidnumber.com. Agents are ready. KB is pre-populated. The AI knows what industry they're in and speaks the right language from conversation one.

The Economics: Why Shared Infrastructure

The alternative to shared multi-tenancy is per-tenant infrastructure: separate databases, separate containers, separate deployments. Some enterprise SaaS products do this. Here's why we don't:

At 1,000 tenants, shared infrastructure costs 35x less. The engineering complexity of application-layer isolation is the price you pay for this efficiency. For a platform serving SMBs at $89-$499/month, per-tenant infrastructure would make the economics impossible.

What We Learned

1. Application-layer filtering is the real enforcement. RLS is defense-in-depth, not the primary mechanism. Developers need to see company_id in every query explicitly. Invisible database policies don't prevent bugs — they catch them after the fact.
2. AI adds three new isolation boundaries. Memory, credentials, and knowledge bases don't exist in traditional SaaS. Each needs its own isolation pattern: Redis key namespacing for memory, Fernet encryption for credentials, per-company cloning for KB.
3. Industry templates solve the cold-start problem. A new dental practice doesn't start with an empty AI. It starts with dental terminology, dental services, dental FAQ — from one of 52 industry templates. The AI is useful from conversation one.
4. Provisioning speed matters. 2-5 seconds to create a fully-configured tenant with agents, KB, budget, and site. If provisioning took minutes, the signup flow breaks. Everything is synchronous, single-transaction.
5. Budget isolation prevents noisy neighbors. One company's viral chatbot can't consume platform-wide AI budget because every company has their own Redis-tracked limits. CognitiveLimiter enforces this at request time.

Multi-Tenancy Is the Hardest Part

Building an AI agent is straightforward. Building an AI agent that serves one company is straightforward. Building AI agents that serve hundreds of companies on shared infrastructure — where every conversation, every memory, every API key, every knowledge base entry, and every dollar of AI spend is perfectly isolated — is the hard part.

That's why multi-tenancy is foundational to Solid#, not a feature bolted on later. Every table has company_id. Every query filters by it. Every Redis key is namespaced. Every agent config, every conversation, every memory — scoped.

This is what makes it infrastructure, not just software.