Sovereign Private GPT
Your teams already use AI, and what really matters is where they do it. With no internal tool, people paste company data into ChatGPT, Copilot or Gemini. That is Shadow AI, and the business loses control of its data. Private GPT brings those uses back under your governance, with a centralized sovereign AI hosted in France, chat and retrieval over your own documents included.
Private GPT combines two open source components, operated for you by Bunker:
- Open WebUI: the chat interface (ChatGPT equivalent), with RAG to query your internal documents.
- vLLM Semantic Router: the AI router that receives each request and directs it, by intent and by reasoning effort, to a local LLM backend or a sovereign cloud AI.
There is no license fee on either piece of software. Bunker charges for managed sovereign hosting and enablement.
Who it is for​
Organizations that want to offer an AI assistant to their teams without exposing their data to platforms subject to US law, and that handle sensitive documents (accounting and legal firms, HR, healthcare, public sector, mid-market companies). RAG lets you query an internal document base, whose documents stay stored inside Bunker.
Architecture​
In practice, the WebUI talks only to the Envoy gateway (OpenAI-compatible API), which consults the Semantic Router (ExtProc) to classify the request and pick the route. The PII guard detects personal data; a request containing a disallowed PII type is blocked before any cloud call (fail-closed), never passed through in clear. Only the types on the allowlist (piiTypesAllowed, email addresses by default) can reach the cloud route. Detection is a filter that sharply reduces the risk without removing it, since a classifier can miss personal data it does not recognize. Detections are counted in the router metrics (per-detection logging is on the roadmap). RAG document embeddings are computed on the private backend, and both documents and their vectors (pgvector) stay stored inside Bunker. When a question takes the general route, the retrieved excerpts that come with it go to Mistral along with it, through the same personal-data filter. To keep every excerpt inside the infrastructure, classify these uses as the sensitive route (private backend, fail-closed).
The Bunker cluster does not serve model weights (no GPU). Both services (Open WebUI, Semantic Router) are CPU workloads. Heavy inference is delegated to external backends, a local/private backend that you provide (chat and embeddings) and Mistral La Plateforme (sovereign EU cloud) on your account.
What the routing does​
| Request type | Route | Behavior |
|---|---|---|
| Sensitive | Private backend | Fail-closed: if the private backend is unavailable, the request fails with an explicit error. Never falls back to the cloud. |
| General | Mistral La Plateforme (EU) | PII detected; a request carrying a disallowed PII type is blocked before sending (fail-closed). Model choice (small/large) by reasoning effort. |
| RAG (documents) | Embeddings on the private backend | Documents and their vectors stay stored inside Bunker (pgvector on PostgreSQL). Retrieved excerpts follow the route of the question. |
No failure is silent. An unavailable private backend on a sensitive route stays visible (clear error), and any request carrying disallowed PII is blocked before the cloud rather than leaked.
Governance: your AI policy becomes configuration​
An AI policy written in a document only applies if someone enforces it. In Private GPT, its rules become service settings, applied to every request.
- Sensitive data stays in-house. The private route runs fail-closed (
semanticRouter.failClosed). If the private backend goes down, the request fails instead of falling back to the cloud. - No disallowed personal data in an external AI. The PII guard (
semanticRouter.guards.pii.*) blocks, before the cloud call, any personal-data type missing from the allowlist. - Every team is identifiable. We recommend a separate API key per team or per application as soon as tools other than the chat connect to the gateway. Per-team keys and per-key spend caps are not included as standard in the current version, and are set up on request, at deployment. As standard, consumption is tracked per route and per model.
- Switching model or provider is a configuration change. It happens in the tenant
values. Open WebUI, like any application calling an OpenAI-compatible API, needs no change.
The same approach extends to every AI use in the company, beyond chat, with a sovereign AI gateway placed in front of all your applications.
CISO & IT recommendations​
Private GPT only guarantees data control if access to public AIs is blocked on the network side. Without that blocking, staff route around the tool and expose company data. Bunker recommends that CISOs and IT teams:
- Block, at the DNS/proxy level, the domains of public consumer AIs (for example
chatgpt.com,chat.openai.com,gemini.google.com,claude.ai) and redirect usage to the internal Private GPT. - Support the switch with training, so teams adopt the internal tool rather than route around it.
- Monitor residual access attempts to public AIs.
This blocking is the client's network responsibility. Bunker provides the recommended list of domains and change-management support, but does not operate the blocking.
A Private GPT deployed without blocking public AIs leaves the door open to Shadow AI. The technical side (sovereign hosting, personal-data detection and blocking) is not enough on its own. It must be paired with a network policy and change management.
Configuration reference (values.yaml)​
Every chart parameter carries a description, a unit, a default and bounds; the exhaustive reference is the commented values.yaml of the chart. Key options:
Authentication​
| Parameter | Type | Default | Description |
|---|---|---|---|
auth.sso.enabled | boolean | false | Enables OIDC SSO. Native Open WebUI auth by default. |
auth.oidc.issuerUrl | url | none | Issuer of the client's IdP (e.g. your Nextcloud). |
auth.oidc.scopes | string | openid email profile | Requested OIDC scopes. |
Retention & deletion​
| Parameter | Type | Default | Description |
|---|---|---|---|
retention.chatsDays | integer ≥ 0 | 0 | Purge chats after N days. 0 = no purge. |
retention.ragDocsDays | integer ≥ 0 | 0 | Purge RAG documents after N days. 0 = no purge. |
retention.piiLogsDays | integer ≥ 0 | 90 | Retention of personal-data detection logs. |
These parameters describe the target retention policy. In this release, automatic purge is not active yet. Deletion is manual and documented; scheduled purge and self-service deletion are on the roadmap.
Router, guards and backends​
| Parameter | Type | Default | Description |
|---|---|---|---|
semanticRouter.replicaCount | integer 1..4 | 2 | The router is in the path of every request; at least 2 replicas. |
semanticRouter.guards.pii.enabled | boolean | true | Enables PII detection on the cloud route. |
semanticRouter.guards.pii.piiTypesAllowed | list | [EMAIL_ADDRESS] | PII types tolerated toward the cloud. A type detected outside this list blocks the cloud route (detection + blocking, no redaction). |
semanticRouter.guards.pii.threshold | float 0..1 | 0.9 | Confidence threshold of the PII classifier. |
semanticRouter.failClosed | boolean | true | Global fail-closed: PII guard or private backend unavailable on a sensitive route = explicit error, never a cloud fallback. |
semanticRouter.modelCache.size | GB (≥ 30) | 40Gi | Local model cache for the router (avoids re-download on restart). |
Open WebUI and RAG​
| Parameter | Type | Default | Description |
|---|---|---|---|
openWebui.rag.vectorStore | enum | pgvector | Vector store, on PostgreSQL CNPG. |
openWebui.rag.embedding.model | string | bge-m3 | Embedding model, served by the private backend. |
openWebui.limits.maxFileSizeMb | integer 1..1024 | 200 | Maximum size per file. Rejected beyond. |
openWebui.limits.maxFileCount | integer 1..1000 | 20 | Maximum number of files per upload. Rejected beyond. |
openWebui.limits.maxDocsPerUser | integer ≥ 1 | 2000 | Per-user document cap. Advisory (roadmap). |
openWebui.limits.maxCorpusMb | integer ≥ 1 | 20480 | Total corpus size (20 GB by default). Advisory (roadmap). |
openWebui.limits.allowedTypes | list | [pdf, docx, txt, md, csv, xlsx] | Accepted file types. Advisory (roadmap). |
Only the per-file size (maxFileSizeMb) and the file count per upload (maxFileCount) are enforced by the platform, with rejection beyond the limit. The per-user document cap, total corpus size and allowed file types (allowedTypes) are advisory in this release (product-side enforcement is on the roadmap).
GDPR compliance​
A tool that processes team conversations and an internal document base handles personal data. Private GPT is designed to stay under GDPR, with EU residency end to end. Before a real tenant goes to production, Bunker applies a go-live gate:
- Mistral DPA template: the client signs a data-processing agreement (art. 28 GDPR) with Mistral for the cloud leg, with verified EU residency.
- Bunker → client DPA template: Bunker is a processor (it hosts chats and RAG documents). A data-processing agreement is provided as a template.
- Processing register (template): purposes, data categories, retention periods (aligned with
retention.*), recipients (private backend, Mistral). - GDPR notices in the interface and documentation: user information, legal basis, rights.
- EU residency verification of the Mistral endpoint and of backups (MinIO in France).
Bunker acts as a processor. You remain the data controller, which means choosing the legal basis, setting retention periods, handling data-subject rights requests and keeping your register of processing activities. The templates above are a starting point.
If the assistant processes sensitive data or data about vulnerable people (minors, learners, patients), a data protection impact assessment (DPIA, art. 35 GDPR) is likely required. It is the controller's responsibility, and Bunker provides the technical documentation needed to carry it out.
The service is hosted in France by a French company. We never use your conversations or documents to train a model, and our subprocessors are listed in the data-processing agreement (art. 28 GDPR).
The client provides its own Mistral API key and pays for its tokens directly, on its own account. Bunker does not re-bill cloud usage.
The cloud leg is billed per token. High-volume tasks that need little reasoning (OCR, classification, rewording) usually cost less on the private backend, which has a fixed monthly cost. Measure tokens per document during the pilot before deciding, because at low volume per-token billing can remain the cheaper option.
What the managed service includes​
- Managed sovereign hosting of both services on the French Bunker cloud.
- Support, updates, backups and monitoring. Uploaded RAG documents are backed up; see the external backups guide.
- Encrypted secrets (client's Mistral key, private backend key, PostgreSQL credentials, OIDC) managed via SealedSecrets.
- Managed database PostgreSQL (CNPG, pgvector); see the managed PostgreSQL guide.
- Private backend access available over a sovereign tunnel; see the Headscale Zero Trust VPN.
The IT & AI diagnostic, leadership enablement (executive team and IT department) and training are delivered with our partners NoBullshit Conseil and Netir Academy, as a quote, outside the plan.
Sovereignty​
The cloud leg goes only through Mistral La Plateforme (EU), on the client's account, with no US provider by default. RAG embeddings are computed locally, and documents stay stored inside Bunker. Both pieces of software are open source, so you can bring the service back onto your own infrastructure.
Getting started​
- Subscribe from the Private GPT page or the pricing page.
- After subscription, onboarding starts with the Mistral key from your account, the endpoint of your private backend (over a sovereign tunnel), the DPA signature and validation of the GDPR go-live gate.
- Bunker deploys, wires and monitors both services. You access the chat interface and start querying your documents.
Operations (runbook)​
Routine operations of the managed service, handled by the Bunker team.
- Deploy / update:
./deploy.sh <org> [env](idempotent). A version bump changes the image tags in thevalues(Open WebUI, Semantic Router, Envoy) and re-runsdeploy.sh; never a directhelm upgrade. - Rollback:
helm rollback <release>returns to the previous revision. For the database, point-in-time CloudNativePG restore (PITR) from the S3 backup. - Backup / restore: Postgres is continuously backed up to S3 (MinIO, EU residency) via CloudNativePG; uploaded RAG documents are backed up by velero. Restore relies on those two sources.
- SSO switch: set
auth.sso.enabledand fill theauth.oidcblock (issuer, client, secret) in the tenantvalues, then re-rundeploy.sh. Reverting to native auth setsauth.sso.enabledback tofalse. - Monitoring: Semantic Router and Envoy metrics are scraped and wired to managed Grafana; alerts on guard CPU load and blocked (fail-closed) requests.