Full-Stack AI14 min readBy Petrică RădanReference guide
Multi-Tenant AI SaaS Isolation with Gemini, Cloud Run & AlloyDB
How to keep tenants apart in one Cloud Run service, one AlloyDB cluster and one Gemini project: row-level security, filtered vector search, pooled connections, caches and agent memory
A multi-tenant AI SaaS on Google Cloud usually runs every customer through the same Cloud Run service, the same AlloyDB for PostgreSQL cluster and the same Gemini project, because a cluster per customer is expensive to run and slow to onboard. Sharing works only if isolation holds at every layer a request touches: the identity that names the tenant, the request context inside a concurrent container, the pooled database connection, the vector search, the model's caches and the agent's memory.
Each layer fails differently. A tenant ID taken from a client header can be forged. A tenant setting left on a pooled connection is inherited by the next request. A vector search that filters after the nearest-neighbour step returns too few rows for small tenants. A context cache or memory key without a tenant prefix can be read by another customer. The sections below take these one at a time, with the Google Cloud and PostgreSQL features that address each and the limits of those features.
What does a shared-cluster multi-tenant architecture look like?
Stateless API and agent code runs on Cloud Run; relational data, embeddings and long-term agent memory live in AlloyDB; Gemini is called through Vertex AI (Google now calls it Gemini Enterprise Agent Platform). Every request authenticates at the edge, and the tenant it acts for comes from a verified token claim that is carried through compute, storage and model calls.
Google publishes a reference architecture for this shape, the Multi-tenant agentic AI system in the Cloud Architecture Center (first published June 2026). On October 16, 2026 Google added security guidance for governing agents and tools with Agent Identity, Agent Gateway, Agent Registry and Model Armor. Cloud Run reaches AlloyDB and Memorystore over Direct VPC egress, and each service runs as its own minimally scoped service account. A VPC Service Controls perimeter around the project limits data exfiltration through Google APIs; it does not separate one tenant from another inside the project, which is the application's job.
| Architecture Layer | Google Cloud Service | Multi-Tenant Isolation Pattern | Primary Failure Mode Prevented |
|---|---|---|---|
| Ingress and Governance | Identity Platform, Agent Gateway, Model Armor | JWT tenant claims, Agent Identity scoping, prompt injection filtering | Spoofed tenant headers and adversarial prompt attacks |
| Stateless Compute | Cloud Run, MCP Servers | Tenant ID in a request-scoped context variable; tenant setting scoped to one transaction | Concurrent coroutine bleed and connection pool session leakage |
| Relational and Vector Store | AlloyDB for PostgreSQL (ScaNN) | Row-level security (or parameterized secure views) plus an explicit tenant filter in vector queries | Cross-tenant reads and post-filter recall loss for small tenants |
| Agent Memory and Inference | Vertex AI Gemini, Memorystore for Valkey | Caching and logging settings chosen deliberately, tenant-prefixed cache handles and memory keys | Cache, memory or log entries readable outside their tenant |
How do you isolate tenant rows in AlloyDB?
Put the tenant check inside the database, with PostgreSQL row-level security (RLS) or AlloyDB parameterized secure views (PSV), so that a missing WHERE clause in application code or a malformed agent tool call cannot read or change another tenant's rows.
PostgreSQL Row-Level Security and Composite Indexing
With RLS, every tenant-owned table, including embedding and agent memory tables, gets a non-null tenant_id column and a policy that compares it with a setting the application writes at the start of each transaction. PostgreSQL then adds that condition to every SELECT, UPDATE and DELETE on the table, and a WITH CHECK clause (implied by a USING-only policy) rejects INSERTs and UPDATEs that would write rows for another tenant.
RLS has bypass rules that decide which database role your application may use. Table owners normally bypass RLS unless the table is set to FORCE ROW LEVEL SECURITY, and superusers and roles with the BYPASSRLS attribute always bypass it, FORCE or not. The application must therefore connect as a role that does not own the tables and has NOSUPERUSER and NOBYPASSRLS, while migrations run under a separate owner role. Views are a further trap: a view runs with its owner's privileges, so a view owned by the table owner skips RLS unless it is created WITH (security_invoker = true) on PostgreSQL 15 or later.
Run SELECT rolname, rolsuper, rolbypassrls FROM pg_roles WHERE rolname = current_user; from the application's own connection in CI. If either flag is true, or the role owns the tenant tables without FORCE ROW LEVEL SECURITY, every policy in this article is silently skipped.
RLS needs indexes that lead with tenant_id. In one published benchmark (MVP Factory, 1 million rows across 10,000 tenants), a query that took 0.8 ms rose to 120 ms with RLS and no suitable index, because PostgreSQL fell back to sequential scans; a composite index leading with tenant_id brought it back to 1.2 ms, and a covering index to 0.9 ms. The same author reports policy evaluation of about 0.3 ms on 50 million rows under 500 concurrent connections, with the transaction-scoped setting adding under 0.1 ms. These are one author's measurements; repeat them on your own schema and data before relying on them.
Parameterized secure views (Preview)
AlloyDB parameterized secure views are an alternative that does not depend on a session setting. You install the parameterized_views extension (after enabling the parameterized_views.enabled flag), define views WITH (security_barrier) that reference named application parameters such as $@app_end_userid, and grant a single application role SELECT on the views but not on the underlying tables. Queries then pass the parameter explicitly, either with EXECUTE ... WITH VIEW PARAMETERS (app_end_userid = ...) on a statement prepared AS RESTRICTED, or through parameterized_views.execute_parameterized_query(). The feature is in Preview under Google's Pre-GA terms, so check its status and limitations before you rely on it for a production tenant boundary.
Embeddings can stay in the same isolated tables. Google describes AlloyDB generating embeddings for text columns automatically, up to 3,000 per second with incremental refresh, and a reference agent memory design that indexes user_id, project_id and scope columns and isolates memory with RLS or PSV (Google Cloud Blog, Implementing long-term AI agent memory in AlloyDB and Memorystore).
How do you keep filtered vector search accurate for each tenant?
Approximate nearest-neighbour indexes and tenant filters interact badly. If the index returns the global top-K neighbours and the tenant filter is applied afterwards, a tenant that owns 0.1% of the vectors often gets back fewer rows than it asked for, or none.
With pgvector's HNSW index this is the classic over-filtering problem; pgvector 0.8.0 added iterative index scans (hnsw.iterative_scan) that keep scanning until enough rows pass the filter, at some cost in latency. AlloyDB's ScaNN index takes a different route and applies the filter during the index scan.
ScaNN Inline Filtering and Adaptive Filtering
AlloyDB offers two mechanisms, and the second is still in Preview:
- Inline filtering (ScaNN only): AlloyDB uses ordinary indexes on the filter columns, such as a B-tree on tenant_id, to build a bitmap of matching rows and computes vector distances only for those rows during the ScaNN scan. Google recommends a B-tree index on the columns used in filters.
- Adaptive filtering (Preview): AlloyDB watches filter selectivity at run time and switches between pre-filtering and inline filtering. It is enabled with the scann.enable_preview_features database flag, which turns on every ScaNN preview feature at once.
Put the tenant condition in the vector query itself as well as in the RLS policy. Google's documentation describes inline filtering for WHERE-clause filters backed by an index; it does not say whether the condition RLS adds is used the same way. An explicit WHERE tenant_id = ... gives the planner an indexable filter, RLS remains the backstop if the clause is ever dropped, and EXPLAIN ANALYZE shows which plan you actually got.
Google's own figures for ScaNN against pgvector HNSW are up to 10x faster filtered queries, up to 6x faster unfiltered queries and up to 16x faster index builds, with support for over 10 billion vectors (AlloyDB AI vector search overview). Google also reports that asymmetric hashing quantization gives 4x more compression than scalar quantization (SQ8) and up to 10x higher write throughput than HNSW. These are vendor benchmarks; measure recall and latency on your own tenant distribution. For very large tables, the ScaNN num_leaves parameter goes up to 100 million on PostgreSQL 17 and later and 30 million on PostgreSQL 16 and earlier.
Noisy neighbours: isolating agent compute
Data isolation does not stop one tenant's agents from saturating the database and slowing everyone else. Google describes an AlloyDB agentic database architecture that serves agent reads from lightweight, isolated microVMs running AlloyDB compute on the shared storage layer, so agent bursts do not compete with the primary instance. In Google's test, index lookup throughput rose from 3.9K to 41K queries per second going from one to ten agent compute nodes without affecting primary I/O (Google Cloud Blog, AlloyDB's agentic database architecture). This addresses performance isolation, not data isolation; the RLS and query rules above still apply on every node.
| Vector index metric (Google's figures) | AlloyDB ScaNN | pgvector HNSW |
|---|---|---|
| Filtered Vector Search Speed | Up to 10x faster (inline filtering) | Baseline; post-filtering can return too few rows unless iterative scans are on |
| Standard Vector Search Speed | Up to 6x faster | Baseline |
| Index Build Speed | Up to 16x faster | Baseline |
| Write Throughput and Compression | Up to 10x higher writes; 4x compression with AH vs SQ8 | Baseline HNSW graph update overhead |
How do you stop tenant context leaking between Cloud Run requests?
A Cloud Run instance handles many requests at once, so the tenant must never be stored in a global variable or in a session-level database setting. It belongs in request-scoped state in the application and in transaction-scoped state in the database.
Request-scoped tenant context and connection pooling
Take the tenant only from a verified identity: a claim in a JWT whose signature, issuer, audience and expiry your middleware has checked (Identity Platform tokens carry a tenant ID), or a header set by a gateway that verified the token and that clients cannot reach around. Never accept a tenant ID from a request body, query string or client-set header. The middleware writes the verified ID to a request-scoped context variable (Python's contextvars.ContextVar) and the database layer writes it into each transaction when it checks out a pooled connection.
Pools also need sizing. At 50 Cloud Run instances, a SQLAlchemy pool of pool_size=5 with max_overflow=5 can open 500 connections; with a max_connections of 100, the database runs out at 20 instances (Madhav's blog, PostgreSQL connection pooling on Cloud Run). Keep max instances times per-instance pool size under your instance's max_connections, or put a transaction-mode pooler in front, which only works because the tenant setting is transaction-local.
Never pass the tenant with a session-level SET on a pooled connection. Use set_config('app.current_tenant', :tid, true) or SET LOCAL inside an explicit transaction, so COMMIT or ROLLBACK clears it, and write the policy to fail closed: when the setting is missing or empty, it must match no rows.
The schema and SQLAlchemy code below combine these rules: a fail-closed policy, a request-scoped tenant, a transaction-scoped setting and an explicit tenant filter on the vector query.
1# Schema (run as the owner/migration role, not the application role):
2# ALTER TABLE tenant_documents ENABLE ROW LEVEL SECURITY;
3# ALTER TABLE tenant_documents FORCE ROW LEVEL SECURITY;
4# CREATE INDEX idx_tenant_docs_tenant ON tenant_documents (tenant_id);
5# CREATE INDEX idx_tenant_docs_scann ON tenant_documents
6# USING scann (embedding cosine) WITH (num_leaves = ...);
7# CREATE POLICY tenant_isolation_policy ON tenant_documents
8# USING (tenant_id = NULLIF(current_setting('app.current_tenant', true), '')::uuid);
9# CREATE ROLE app_user LOGIN NOSUPERUSER NOBYPASSRLS; -- not the table owner
10# GRANT SELECT, INSERT, UPDATE, DELETE ON tenant_documents TO app_user;
11
12from contextvars import ContextVar
13from contextlib import asynccontextmanager
14from sqlalchemy import text
15from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
16
17# Set by middleware from a verified token claim, never from client input
18current_tenant_ctx: ContextVar[str] = ContextVar("current_tenant_ctx")
19
20@asynccontextmanager
21async def get_tenant_db_session(
22 session_factory: async_sessionmaker[AsyncSession],
23):
24 tenant_id = current_tenant_ctx.get(None)
25 if not tenant_id:
26 raise PermissionError("Missing verified tenant context for database access.")
27
28 async with session_factory() as session:
29 async with session.begin():
30 # is_local=true: the setting ends with this transaction,
31 # so COMMIT or ROLLBACK clears it before the connection returns to the pool.
32 await session.execute(
33 text("SELECT set_config('app.current_tenant', :tid, true)"),
34 {"tid": tenant_id},
35 )
36 yield session
37
38async def retrieve_tenant_chunks(
39 session_factory: async_sessionmaker[AsyncSession],
40 query_vector_literal: str,
41 row_limit: int,
42):
43 async with get_tenant_db_session(session_factory) as session:
44 # The explicit tenant filter lets ScaNN inline filtering use
45 # idx_tenant_docs_tenant; RLS still blocks other tenants if it is dropped.
46 sql = text("""
47 SELECT document_id, chunk_text
48 FROM tenant_documents
49 WHERE tenant_id = CAST(:tid AS uuid)
50 ORDER BY embedding <=> CAST(:qvec AS vector)
51 LIMIT :lim
52 """)
53 result = await session.execute(
54 sql,
55 {"tid": current_tenant_ctx.get(), "qvec": query_vector_literal, "lim": row_limit},
56 )
57 return result.fetchall()Prove it: cross-tenant tests
Isolation claims need tests that try to break them. The pytest suite below runs against a disposable PostgreSQL 15+ database (RLS behaves the same as on AlloyDB) seeded with rows for two synthetic tenants, connecting as the application role. A pool of two connections forces both tenants through the same connections. The tests cover the application role's privileges, interleaved concurrent requests, connections reused after committed and failed requests, a cancelled request and a request with no tenant. We ran it against PostgreSQL 17: it passes with the code above, and the committed-request test fails if set_config is switched to a session-level setting, which is the leak it exists to catch.
1# tests/test_tenant_isolation.py (pytest-asyncio, asyncio_mode = "auto")
2# Fixture seeded_db_url: fresh PostgreSQL 15+ with the schema above and 50 rows each
3# for TENANT_A and TENANT_B, returning a URL that logs in as app_user.
4import asyncio
5import pytest
6from sqlalchemy import text
7from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
8
9from app.db import current_tenant_ctx, get_tenant_db_session
10from tests.seed import TENANT_A, TENANT_B
11
12@pytest.fixture
13async def factory(seeded_db_url):
14 # Two connections shared by every request: each one is reused across tenants
15 engine = create_async_engine(seeded_db_url, pool_size=2, max_overflow=0)
16 yield async_sessionmaker(engine, expire_on_commit=False)
17 await engine.dispose()
18
19async def tenants_visible(factory, tenant):
20 token = current_tenant_ctx.set(tenant)
21 try:
22 async with get_tenant_db_session(factory) as s:
23 await asyncio.sleep(0) # let other requests run mid-transaction
24 rows = await s.execute(text("SELECT DISTINCT tenant_id::text FROM tenant_documents"))
25 return {r[0] for r in rows}
26 finally:
27 current_tenant_ctx.reset(token)
28
29async def test_app_role_cannot_bypass_rls(factory):
30 async with factory() as s:
31 row = (await s.execute(text(
32 "SELECT rolsuper, rolbypassrls FROM pg_roles WHERE rolname = current_user"
33 ))).one()
34 assert row == (False, False)
35
36async def test_interleaved_requests_see_only_their_tenant(factory):
37 tenants = [TENANT_A, TENANT_B] * 200
38 seen = await asyncio.gather(*(tenants_visible(factory, t) for t in tenants))
39 assert all(s == {t} for t, s in zip(tenants, seen))
40
41async def assert_pool_has_no_tenant(factory):
42 # Hold both pooled connections at once so each one is checked
43 async with factory() as s1, factory() as s2:
44 for s in (s1, s2):
45 assert not await s.scalar(text("SELECT current_setting('app.current_tenant', true)"))
46 assert await s.scalar(text("SELECT count(*) FROM tenant_documents")) == 0 # fails closed
47
48async def test_connections_are_clean_after_committed_requests(factory):
49 await asyncio.gather(tenants_visible(factory, TENANT_A), tenants_visible(factory, TENANT_A))
50 await assert_pool_has_no_tenant(factory)
51
52async def test_connections_are_clean_after_failed_request(factory):
53 token = current_tenant_ctx.set(TENANT_A)
54 with pytest.raises(RuntimeError):
55 async with get_tenant_db_session(factory):
56 raise RuntimeError("handler failed mid-transaction")
57 current_tenant_ctx.reset(token)
58 await assert_pool_has_no_tenant(factory)
59
60async def test_cancelled_request_does_not_leak(factory):
61 async def slow_request():
62 token = current_tenant_ctx.set(TENANT_A)
63 try:
64 async with get_tenant_db_session(factory) as s:
65 await s.execute(text("SELECT pg_sleep(5)"))
66 finally:
67 current_tenant_ctx.reset(token)
68
69 task = asyncio.create_task(slow_request())
70 await asyncio.sleep(0.2)
71 task.cancel()
72 with pytest.raises(asyncio.CancelledError):
73 await task
74 assert await tenants_visible(factory, TENANT_B) == {TENANT_B}
75
76async def test_missing_tenant_is_refused(factory):
77 with pytest.raises(PermissionError):
78 async with get_tenant_db_session(factory):
79 passRun the same suite in CI on every change to the data layer, and add a load test that interleaves synthetic tenants against the deployed Cloud Run service, since the in-process tests do not cover the gateway or the token checks.
How do you isolate Gemini caches, logs and agent memory per tenant?
Database isolation does not cover what the model layer stores: Google's implicit caches, explicit context caches you create, optional request logging, grounding logs and agent memory. Each has its own retention rule, and some must be switched off deliberately.
What Gemini stores, and how to turn it off
Google states that it does not use customer data to train or fine-tune models without permission. Zero data retention is a different matter: Google's zero data retention page says it requires specific actions in several areas, and none of them is a single switch.
- Implicit caching is on by default for supported Gemini models: an in-memory tier and a durable tier stored at rest, both deleted within 24 hours. Turn off the durable tier (retentionType EPHEMERAL) or both (disableCache true) per project through the cacheConfig endpoint.
- Request-response logging is off by default; leave it off. It is also unavailable inside a VPC Service Controls perimeter.
- Abuse monitoring may log prompts for up to 90 days; customers under a Cloud Master Agreement are exempt by default.
- Grounding with Google Search keeps data for up to three days and cannot be disabled; Google points to Web Grounding for Enterprise for zero retention. Grounding with Google Maps keeps data for 30 days.
- Live API session resumption stores data for up to 24 hours; it is off by default. On the Interactions API, set store to false, because it defaults to true.
Explicit context caches are different: you create them, and they hold tenant documents until their TTL expires or you delete them. Treat each one as tenant data at rest. Record every CachedContent resource name in a table keyed by tenant, check in middleware that the cache belongs to the requesting tenant before passing it to a Gemini call, set short TTLs and delete a tenant's caches when they offboard. A system prompt that contains no customer data can be shared across tenants. Google's 2025 announcement put cached input tokens at 10% of the standard input price for Gemini 2.5 and later models; check current pricing.
Two-Tier Multi-Tenant Agent Memory
Agents need conversation history without resending whole transcripts. Google's reference design uses Memorystore for Valkey for short-term memory and AlloyDB for long-term memory, and Google reports up to 70% lower token spend from the two-tier approach (Google Cloud Blog, Implementing long-term AI agent memory in AlloyDB and Memorystore).
- Short-term tier: Cloud Run reads and writes the active conversation in Memorystore for Valkey over Direct VPC egress, with keys prefixed by tenant (tenant:{tenant_id}:session:{session_id}) and a TTL on every key. Valkey has no row-level security, so the prefix is enforced only by your code: build keys in one function that reads the request-scoped tenant, never from request input.
- Long-term tier: background workers summarize finished sessions with Gemini and write the results to AlloyDB tables that carry tenant_id and the same RLS policies as the rest of the data. Define retention for these memories and delete them with the tenant's other data on offboarding.
In what order should you build it?
Work from the data outward, and add the tests before the first real tenant arrives:
- Database first: a non-null tenant_id on every tenant table (including embeddings and memory), RLS enabled and forced, an application role that is not the owner and has NOBYPASSRLS, and indexes leading with tenant_id next to the ScaNN index.
- Identity and context: tenant taken only from a verified token claim, held in a request-scoped context variable and written per transaction with set_config(..., true).
- Vector queries: an explicit tenant filter in every similarity query, checked with EXPLAIN ANALYZE.
- Model layer: choose caching, logging and grounding settings against Google's zero data retention list, and key explicit caches and Valkey memory by tenant.
- Network: Direct VPC egress, per-service accounts and a VPC Service Controls perimeter against exfiltration.
- Tests: the cross-tenant suite above in CI, plus a load test that interleaves synthetic tenants against the deployed service.
Frequently asked questions
Does FORCE ROW LEVEL SECURITY apply to superusers?
No. FORCE ROW LEVEL SECURITY makes the table owner subject to RLS, but superusers and roles with the BYPASSRLS attribute always bypass it. Connect the application as a separate role with NOSUPERUSER and NOBYPASSRLS that does not own the tables.
Is SET LOCAL safe with a connection pool?
Yes, if it runs inside an explicit transaction: SET LOCAL and set_config(name, value, true) end with the transaction, so the next request on that connection does not inherit the tenant. A plain SET lasts for the whole session and does leak through a pool.
Does Gemini on Vertex AI keep prompts?
Google says it does not train on customer data without permission, but implicit caching keeps data for up to 24 hours by default, abuse monitoring may log prompts for up to 90 days unless you are exempt, and Grounding with Google Search keeps data for up to three days. Zero data retention requires changing or avoiding each of these.
Do I need a separate AlloyDB cluster per tenant?
Not for isolation alone: RLS, explicit tenant filters and tested pooling keep tenants apart in a shared cluster. Separate clusters are still worth it when a contract requires physical separation, a customer needs its own encryption keys or region, or one tenant's load would dominate.
Sources
- Google Cloud Architecture Center, Multi-tenant agentic AI system (opens in a new tab)
- PostgreSQL documentation, Row Security Policies (opens in a new tab)
- AlloyDB documentation, Parameterized secure views overview (Preview) (opens in a new tab)
- AlloyDB documentation, Filtered vector search overview (opens in a new tab)
- AlloyDB documentation, AlloyDB AI vector search overview (opens in a new tab)
- AlloyDB documentation, Best practices for tuning ScaNN indexes (opens in a new tab)
- pgvector README, Iterative index scans (opens in a new tab)
- Google Cloud documentation, Zero data retention (Gemini Enterprise Agent Platform) (opens in a new tab)
- Google Cloud documentation, VPC Service Controls with Vertex AI (opens in a new tab)
- Google Cloud Blog, Implementing long-term AI agent memory in AlloyDB and Memorystore (2026) (opens in a new tab)
- Google Cloud Blog, AlloyDB's agentic database architecture (2026) (opens in a new tab)
- Google Developer forums, Acceleration of ScaNN with the columnar engine in AlloyDB (2025) (opens in a new tab)
- Google Cloud Blog, Vertex AI context caching (2025) (opens in a new tab)
- MVP Factory, PostgreSQL RLS: your last defense against tenant data leaks (2026) (opens in a new tab)
- MVP Factory on DEV Community, PostgreSQL Row-Level Security for Multi-Tenant SaaS (2026) (opens in a new tab)
- Madhav's blog, PostgreSQL connection pooling on Cloud Run (2026) (opens in a new tab)