Sovereign Private GPT
Your teams already use AI, and what really matters is where they do it. With no internal tool, people paste company data into ChatGPT, Copilot or Gemini. That is Shadow AI, and the business loses control of its data. Private GPT brings those uses back under your governance, with a centralized sovereign AI hosted in France, chat and retrieval over your own documents included.
Private GPT combines two open source components, operated for you by Bunker:
- Open WebUI: the chat interface (ChatGPT equivalent), with RAG to query your internal documents.
- vLLM Semantic Router: the AI router that receives each request and directs it, by intent and by reasoning effort, to a local LLM backend or a sovereign cloud AI.
There is no license fee on either piece of software. Bunker charges for managed sovereign hosting and enablement.
Who it is for
Organizations that want to offer an AI assistant to their teams without exposing their data to platforms subject to US law, and that handle sensitive documents (accounting and legal firms, HR, healthcare, public sector, mid-market companies). RAG lets you query an internal document base; documents never leave the Bunker infrastructure.
Architecture
In practice, the WebUI talks only to the Envoy gateway (OpenAI-compatible API), which consults the Semantic Router (ExtProc) to classify the request and pick the route. The PII guard detects personal data; a request containing a disallowed PII type is blocked before any cloud call (fail-closed), never passed through in clear. No personal data leaves for the cloud. Detections are counted in the router metrics (per-detection logging is on the roadmap). RAG document embeddings are computed on the private backend, and documents never leave for the cloud.
The Bunker cluster does not serve model weights (no GPU). Both services (Open WebUI, Semantic Router) are CPU workloads. Heavy inference is delegated to external backends, a local/private backend that you provide (chat and embeddings) and Mistral La Plateforme (sovereign EU cloud) on your account.
What the routing does
| Request type | Route | Behavior |
|---|---|---|
| Sensitive | Private backend | Fail-closed: if the private backend is unavailable, the request fails with an explicit error. Never falls back to the cloud. |
| General | Mistral La Plateforme (EU) | PII detected; a request carrying a disallowed PII type is blocked before sending (fail-closed). Model choice (small/large) by reasoning effort. |
| RAG (documents) | Embeddings on the private backend | Documents and their vectors stay inside Bunker (pgvector on PostgreSQL). |
No failure is silent. An unavailable private backend on a sensitive route stays visible (clear error), and any request carrying disallowed PII is blocked before the cloud rather than leaked.
CISO & IT recommendations
Private GPT only guarantees data control if access to public AIs is blocked on the network side. Without that blocking, staff route around the tool and expose company data. Bunker recommends that CISOs and IT teams:
- Block, at the DNS/proxy level, the domains of public consumer AIs (for example
chatgpt.com,chat.openai.com,gemini.google.com,claude.ai) and redirect usage to the internal Private GPT. - Support the switch with training, so teams adopt the internal tool rather than route around it.
- Monitor residual access attempts to public AIs.
This blocking is the client's network responsibility. Bunker provides the recommended list of domains and change-management support, but does not operate the blocking.
A Private GPT deployed without blocking public AIs leaves the door open to Shadow AI. The technical side (sovereign hosting, personal-data detection and blocking) is not enough on its own. It must be paired with a network policy and change management.
Configuration reference (values.yaml)
Every chart parameter carries a description, a unit, a default and bounds; the exhaustive reference is the commented values.yaml of the chart. Key options:
Authentication
| Parameter | Type | Default | Description |
|---|---|---|---|
auth.sso.enabled | boolean | false | Enables OIDC SSO. Native Open WebUI auth by default. |
auth.oidc.issuerUrl | url | none | Issuer of the client's IdP (e.g. your Nextcloud). |
auth.oidc.scopes | string | openid email profile | Requested OIDC scopes. |
Retention & deletion
| Parameter | Type | Default | Description |
|---|---|---|---|
retention.chatsDays | integer ≥ 0 | 0 | Purge chats after N days. 0 = no purge. |
retention.ragDocsDays | integer ≥ 0 | 0 | Purge RAG documents after N days. 0 = no purge. |
retention.piiLogsDays | integer ≥ 0 | 90 | Retention of personal-data detection logs. |
These parameters describe the target retention policy. In this release, automatic purge is not active yet. Deletion is manual and documented; scheduled purge and self-service deletion are on the roadmap.
Router, guards and backends
| Parameter | Type | Default | Description |
|---|---|---|---|
semanticRouter.replicaCount | integer 1..4 | 2 | The router is in the path of every request; at least 2 replicas. |
semanticRouter.guards.pii.enabled | boolean | true | Enables PII detection on the cloud route. |
semanticRouter.guards.pii.piiTypesAllowed | list | [EMAIL_ADDRESS] | PII types tolerated toward the cloud. A type detected outside this list blocks the cloud route (detection + blocking, no redaction). |
semanticRouter.guards.pii.threshold | float 0..1 | 0.9 | Confidence threshold of the PII classifier. |
semanticRouter.failClosed | boolean | true | Global fail-closed: PII guard or private backend unavailable on a sensitive route = explicit error, never a cloud fallback. |
semanticRouter.modelCache.size | GB (≥ 30) | 40Gi | Local model cache for the router (avoids re-download on restart). |
Open WebUI and RAG
| Parameter | Type | Default | Description |
|---|---|---|---|
openWebui.rag.vectorStore | enum | pgvector | Vector store, on PostgreSQL CNPG. |
openWebui.rag.embedding.model | string | bge-m3 | Embedding model, served by the private backend. |
openWebui.limits.maxFileSizeMb | integer 1..1024 | 200 | Maximum size per file. Rejected beyond. |
openWebui.limits.maxFileCount | integer 1..1000 | 20 | Maximum number of files per upload. Rejected beyond. |
openWebui.limits.maxDocsPerUser | integer ≥ 1 | 2000 | Per-user document cap. Advisory (roadmap). |
openWebui.limits.maxCorpusMb | integer ≥ 1 | 20480 | Total corpus size (20 GB by default). Advisory (roadmap). |
openWebui.limits.allowedTypes | list | [pdf, docx, txt, md, csv, xlsx] | Accepted file types. Advisory (roadmap). |
Only the per-file size (maxFileSizeMb) and the file count per upload (maxFileCount) are enforced by the platform, with rejection beyond the limit. The per-user document cap, total corpus size and allowed file types (allowedTypes) are advisory in this release (product-side enforcement is on the roadmap).
GDPR compliance
A tool that processes team conversations and an internal document base handles personal data. Private GPT is designed to stay under GDPR, with EU residency end to end. Before a real tenant goes to production, Bunker applies a go-live gate:
- Mistral DPA template: the client signs a data-processing agreement (art. 28 GDPR) with Mistral for the cloud leg, with verified EU residency.
- Bunker → client DPA template: Bunker is a processor (it hosts chats and RAG documents). A data-processing agreement is provided as a template.
- Processing register (template): purposes, data categories, retention periods (aligned with
retention.*), recipients (private backend, Mistral). - GDPR notices in the interface and documentation: user information, legal basis, rights.
- EU residency verification of the Mistral endpoint and of backups (MinIO in France).
The client provides its own Mistral API key and pays for its tokens directly, on its own account. Bunker does not re-bill cloud usage.
What the managed service includes
- Managed sovereign hosting of both services on the French Bunker cloud.
- Support, updates, backups and monitoring. Uploaded RAG documents are backed up; see the external backups guide.
- Encrypted secrets (client's Mistral key, private backend key, PostgreSQL credentials, OIDC) managed via SealedSecrets.
- Managed database PostgreSQL (CNPG, pgvector); see the managed PostgreSQL guide.
- Private backend access available over a sovereign tunnel; see the Headscale Zero Trust VPN.
The IT & AI diagnostic, leadership enablement (executive team and IT department) and training are delivered with our partners NoBullshit Conseil and Netir Academy, as a quote, outside the plan.
Sovereignty
The cloud leg goes only through Mistral La Plateforme (EU), on the client's account, with no US provider by default. RAG embeddings are computed locally, and documents stay inside Bunker. Both pieces of software are open source, so you can bring the service back onto your own infrastructure.
Getting started
- Subscribe from the Private GPT page or the pricing page.
- After subscription, onboarding starts with the Mistral key from your account, the endpoint of your private backend (over a sovereign tunnel), the DPA signature and validation of the GDPR go-live gate.
- Bunker deploys, wires and monitors both services. You access the chat interface and start querying your documents.
Operations (runbook)
Routine operations of the managed service, handled by the Bunker team.
- Deploy / update:
./deploy.sh <org> [env](idempotent). A version bump changes the image tags in thevalues(Open WebUI, Semantic Router, Envoy) and re-runsdeploy.sh; never a directhelm upgrade. - Rollback:
helm rollback <release>returns to the previous revision. For the database, point-in-time CloudNativePG restore (PITR) from the S3 backup. - Backup / restore: Postgres is continuously backed up to S3 (MinIO, EU residency) via CloudNativePG; uploaded RAG documents are backed up by velero. Restore relies on those two sources.
- SSO switch: set
auth.sso.enabledand fill theauth.oidcblock (issuer, client, secret) in the tenantvalues, then re-rundeploy.sh. Reverting to native auth setsauth.sso.enabledback tofalse. - Monitoring: Semantic Router and Envoy metrics are scraped and wired to managed Grafana; alerts on guard CPU load and blocked (fail-closed) requests.