Host a private LLM in Europe

• Data stays in Europe. Your prompts don't pass through a third-party provider outside the EU. That's the difference that unblocks GDPR cases and sensitive data.
• No leakage into someone else's training. A model you host sends nothing to anyone. What you give it stays with you.
• Predictable cost at volume. An API bills per token. Once usage gets serious, a dedicated GPU often costs less than a bill that grows with traffic.
• Re-internalisable. The infrastructure is open source. You can move it onto your own hardware whenever you want, taking your data with you.

Open-weight models now cover most needs. The right choice mostly comes down to size, because size decides the GPU.

• 7 to 9 billion parameters (Mistral 7B, Llama 3 8B): fast, enough for classification, summarization, extraction, an internal chatbot. Run on a single consumer GPU.
• 12 to 34 billion: better reasoning, a good quality/cost balance for RAG or business assistance.
• 70 billion and up: quality close to large proprietary models, but it takes several GPUs or quantization.

Quantization (4-bit, 8-bit) sharply reduces the memory needed at a usually acceptable precision cost. A 7B model goes from roughly 16 GB in fp16 to 6-8 GB in 4-bit; a 70B goes from around 140 GB to about forty.

You pick a model and a GPU size, we deploy inference in Europe, and you get a private API that is OpenAI-compatible: your existing tools work without a rewrite. Everything stays in European datacenters.

No data leaves this boundary. And because the stack is open source, you can ask to bring it onto your own infrastructure without starting over.

To size your private AI, we start from your business needs. The technical tuning (quantization, batching, GPU choice) is our job.

1. Model size. A small model (7 to 9 billion parameters) responds fast and costs little: enough for summarization, classification, or an internal assistant. A larger model reasons better on complex topics, at the cost of more resources. We tune the precision (quantization) to fit the GPU envelope.
2. Context size. This is what the model reads in one go: a short question, or dozens of pages injected to query your documents (RAG). It is the most underestimated factor: a long context consumes a lot of memory and slows down the first response. As soon as you handle long documents, it is often what sizes the deployment.
3. The number of truly concurrent users. How many people query the AI at the same time, at peak. Ten connected colleagues almost never fire ten requests in the same second: this figure is often overestimated.
4. Expected speed. Two things that differ: the delay before the first response, which matters for an interactive exchange, and the generation throughput, which matters for bulk background processing. A live agent and an overnight batch do not have the same needs.

When in doubt, we start with a small model and a modest GPU, measure real usage, and scale up if needed. That is cheaper than oversizing from day one.