The operating question comes before the model comparison
Most discussions about internal assistants start with the model: which one understands German best, which one writes the better code. The question that belongs answered first is a different one: which data reaches the assistant, and where may it be processed. An assistant that accesses internal documents, tickets or source code almost always processes personal data too, at the very least the inputs of the employees.
The guidance published by the DSK, the conference of Germany's data protection authorities, recommends exactly this order: first define the fields of use and the purposes, then select and implement the application. The distinction between closed and open systems also matters: the relevant factors include a limited user group, technical isolation and how inputs are subsequently used.
A pattern that keeps recurring. One department starts an assistant as an experiment. Two other departments find it useful and upload documents. Half a year later the question stands in the room whether a staff appraisal that someone pasted in for summarisation has left the building. The answer depends on a decision nobody consciously made: which provider the requests land with and what that provider's contract says. That is why this decision comes at the beginning and not at the end. It is also the most expensive one to correct, because a change of operating model touches every integration that has been built in the meantime.
First, distinguish between a hosted AI service and operating the model yourself. The provider’s home country and processing region are additional selection criteria. The following sections cover cloud services, European providers as a possible source and self-managed operation.
AI service through a cloud API
An AI service accessed through a cloud API runs the model at the provider. Large cloud platforms such as Microsoft Azure, AWS or Google Cloud and direct model providers such as OpenAI or Anthropic offer this access [3][6][7]. You do not need to operate the computers that run the model yourself. Connecting your documents, access permissions and workflows remains a separate task.
For Azure OpenAI, Microsoft states that prompts and outputs are not passed on to OpenAI or used to train foundation models without consent. The processing location depends on the deployment type: Global deployments can process requests in different geographies, while an EU Data Zone processes them within the defined EU zone. Storage locations and individual features also need to be checked [3].
The applicable regions, retention periods and abuse-monitoring settings depend on the specific service and contract. The provider’s home country or a selected data centre does not answer these questions alone. The assistant’s entire processing chain needs to be reviewed.
European providers as an option
A European provider can also supply a hosted AI service through an API, such as Mistral or a European cloud provider. This is a provider choice within hosted operation. A contracting partner based in the EU does not guarantee that all processing takes place in Europe. Check processing regions, subprocessors and the specific features used, as well as potential third-country transfers. Mistral, for example, describes a US endpoint and possible processing outside the EU for certain features alongside its EU processing [5].
Model choice, additional features and costs vary by provider and service. A pilot with real tasks establishes whether a solution suits summaries, document retrieval or drafting work. The company’s home country alone says little about that.
Operating AI yourself
In self-managed operation, you run open models yourself, in your own data centre or on dedicated servers. Tools such as Ollama allow local execution and integration with applications through an API.
With fully internal inference, inputs and outputs can stay within your own environment. Document processing, embeddings, logging and other integrations must also be configured accordingly. Your own team manages GPU hardware, model updates, availability, access control and interface security. If hosting or other service providers are involved, their roles and data flows also need review. Local operation does not automatically mean secure operation.
Data protection: processing agreements and data flow
With a hosted AI service, the provider processes data on your behalf, which requires a data processing agreement and a review of which data the provider uses for its own purposes, for example abuse monitoring. The DSK guidance additionally requires, among other things, an established legal basis, corporate instead of private accounts, transparency towards employees and restraint in entering personal data.
At the European level, the European Data Protection Board has clarified in Opinion 28/2024 under which conditions AI models themselves count as personal data and what that means for controllers. For selecting a provider this means: the origin of the model and its training data can become a review question too, alongside the operation itself.
The decision combines operating model, provider and processing location. First, establish which data may reach the assistant and who may process it. Then evaluate a hosted AI service, self-managed operation or a combination. A European provider is one possible provider choice, not a separate operating model.
Cost logic: consumption versus base load
Hosted services and self-managed operation also differ in their cost profile. Cloud APIs are often billed by consumption: with little usage, little is due; with intensive usage, costs grow along, and a single team can drive them unexpectedly with one data-hungry use case. Budget limits and a per-team consumption overview therefore belong in place from the start.
Local operation reverses the logic: hardware, power and operating time are due largely independent of usage. That only pays off from a certain utilisation, but then predictably. Whoever does not yet know their utilisation therefore sensibly starts with a consumption-based model and measures before investing in their own hardware.
Comparing operating models and providers
| Criterion | Cloud service, such as Azure | EU provider’s service | Self-managed |
|---|---|---|---|
| Data flow | to the provider; check region and deployment | to the provider; verify the processing region | internal if the entire processing chain is configured accordingly |
| Contractual basis | processing agreement; review potential third-country transfers | processing agreement; review potential third-country transfers | no external processor with fully internal operation |
| Model offering | depends on the service and model access | depends on the service and model access | open models, selection curated yourself |
| Operational effort | low | low | high, own team required |
| Cost profile | grows with consumption | grows with consumption | base load, pays off at high utilisation |
Mixed setups can make sense: the data class, contract and verified processing settings determine which data may reach which model. A gateway can enforce these rules; document retrieval must additionally respect each user's access permissions.
What you can decide afterwards
- Which operating model fits your data classes: a hosted AI service, self-managed operation or a combination, each with verified provider and regional settings.
- Which contracts and reviews come before the start: data processing agreement, third country transfer and the requirements of the DSK guidance.
- Whether you start consumption-based and measure utilisation, or invest directly in your own hardware.
Frequently asked questions
What does an AI service through a cloud API mean?
Your application sends requests through a programming interface to a model operated by the provider. The response returns through the same interface. Where processing occurs and which data is stored need to be checked for each service and its settings.
Is local operation automatically the most privacy-friendly choice?
No. A fully internal processing chain can avoid external data transfers. Access control, interfaces, document processing and logging must be configured accordingly. Data protection and security remain responsibilities of the operating team, including in local deployments.
Can we combine several operating models?
Yes. Data classes, contracts and verified processing settings determine which data may reach which model. A gateway can route requests accordingly. Document retrieval must additionally enforce each user's access permissions.
Do we have to involve the data protection officer before the start?
Yes, and before the provider is chosen. The data processing agreement, the third country review and the DSK requirements depend on the operating model, and a later correction touches every integration that has been built in the meantime.
Sources
- Orientierungshilfe Künstliche Intelligenz und Datenschutz, Version 1.0 (PDF) Datenschutzkonferenz (DSK), 6 May 2024
- Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models European Data Protection Board, 18 December 2024
- Data, privacy, and security for Foundry Models sold by Azure Microsoft Learn
- Ollama Documentation Ollama
- Where do you store my data or my organization’s data? Mistral AI
- Claude API overview Anthropic, direct API access and cloud platforms
- OpenAI API overview OpenAI documentation