How to Train Your Own AI Model on Company Data

Contents

Why the data is the hard part
The six stages
Fine-tuning, retrieval or continued pre-training
The European rules that apply
Where OpenLazarus fits
What success looks like
Questions buyers ask
Sources

Why the data is the hard part

A company that has moved its AI work onto an open-weight model has solved the cost problem but not the knowledge problem. The model knows what every copy of it knows. Revolut, which moved about 2,000 engineers onto a self-hosted GLM model, reports large savings, but nothing in its published account describes training that model on its own code, tickets or decisions⁠1 2. The gains from its earlier Gemini rollout came from access to internal data, mostly by replacing documentation lookups⁠2.

Training a model on company data is the step that makes it different from a competitor's copy. The training itself is no longer the obstacle: tooling such as Hugging Face's TRL and PEFT libraries makes fine-tuning routine⁠3 4. The obstacle is that company knowledge was written for people, not for models. It is spread across mail, documents, tickets, chat threads and system records. Much of it is duplicated, out of date or contradicts itself, and almost none of it is in the form of a question and a correct answer.

In practice the work resembles a library more than a laboratory. Someone has to find what the company knows, discard what is wrong, put the rest in order and keep doing so as the company changes.

The six stages

1. Take an inventory

List every place the company's working knowledge lives and what each holds: mail and shared inboxes, document stores and wikis, ticketing and support systems, team chat, the CRM, the ERP and other systems of record. For each source, note who owns it, how current it is, whether it contains personal data and how it can be exported. The inventory decides what the model can learn. Knowledge that lives only in one person's head or one person's inbox is invisible to any model.

2. Clean it

Remove duplicates, drafts and superseded versions. Resolve conflicts, such as two price lists or two versions of a procedure, by deciding which is authoritative rather than leaving both in. Strip or mask personal data that the model does not need. This stage usually takes longer than training, and it is where most of the quality of the final model is decided.

3. Structure and label it

A model learns from examples of the work it should do. Turn the cleaned material into pairs of a task and a good result: a customer question and the answer the support team would give, a supplier email and the correct follow-up, a ticket and its resolution. Label each example with its source and date so it can be traced and retired later.

4. Build an evaluation set

Before training, set aside a few hundred real tasks with known good answers that the model will never see during training. This is the only way to tell whether the trained model is better than the base model, and whether a later version is better than the last. Without it, every improvement is an impression.

5. Choose a base model

Pick the open-weight model to train on by licence first, then by size. Licences decide what the company may do with the result. Size decides the hardware.

Need	Options reviewed on this site	Why
Plain licence, one GPU	Gemma 4, gpt-oss	Apache 2.0; Gemma 4 26B and 31B fit one 80 GB GPU, gpt-oss-120b fits one 80 GB GPU⁠5 6
European developer	Mistral	Mistral Small 4 is Apache 2.0, with an Axolotl fine-tuning example linked⁠7 8
Frontier scale, permissive licence	DeepSeek	MIT licence for code and weights⁠9
The model Revolut runs	GLM	Together AI lists GLM-5.1 among the models it fine-tunes⁠10
Full transparency	OLMo	Training data, code and checkpoints all published⁠11

6. Train, test and route

For most companies the right method is parameter-efficient fine-tuning. PEFT methods such as LoRA train a small number of extra parameters on top of a frozen base model, which makes training feasible on modest hardware⁠4. QLoRA, which combines LoRA with a 4-bit quantised base model, fine-tuned a 65-billion-parameter model on a single 48 GB GPU⁠12. Google states that Gemma 4 can be fine-tuned on a gaming GPU⁠5, and OpenAI documents fine-tuning gpt-oss-120b on a single H100 node and gpt-oss-20b on consumer hardware⁠6.

Test the trained model against the evaluation set, compare it with the base model and with any frontier model in use, and only then route work to it. Serving software such as vLLM exposes a trained model as an OpenAI-compatible endpoint⁠13, so it can sit behind the same router as every other model.

Fine-tuning, retrieval or continued pre-training

Three methods put company knowledge into an AI system, and they solve different problems.

Method	What changes	Fits best when
Retrieval (RAG)	Nothing in the model; relevant documents are added to each prompt	Facts change often: prices, stock, policies, open tickets
Fine-tuning	A small set of added weights (LoRA) or the full model	The model must use the company's terms, formats and judgement consistently
Continued pre-training	The full model, on large volumes of raw text	A specialised language or domain the base model barely knows; the most expensive option

Retrieval is the quickest to set up and keeps facts current, but the model still reasons like a general model. Fine-tuning teaches behaviour and is cheap with LoRA, but facts learned in training go stale. Most companies end up combining a fine-tuned model with retrieval for live data. Continued pre-training is rarely needed outside unusual domains.

The European rules that apply

Training a model on personal data is processing under the GDPR. The European Data Protection Board's Opinion 28/2024 sets out the main points: a model trained on personal data cannot in all cases be considered anonymous; legitimate interest can be a lawful basis for developing and deploying a model, subject to a three-step test of legitimacy, necessity and balancing; and a company that deploys a model should check that it was lawfully developed⁠14. France's CNIL publishes practical guidance on the legal basis and on a model's status under the GDPR⁠15.

Under the AI Act, fine-tuning an open model usually does not make a company the provider of a new general-purpose model: the Commission's threshold is a modification using more than a third of the original model's training compute⁠16. Our guide to the AI Act and open-source models covers the details.

Where OpenLazarus fits

Of the options on this site, OpenLazarus is the one built for the data work rather than the model. Its AI-Architect engagement builds a data lake that holds all of a company's data, owned by the company rather than its software vendors⁠17. Its AI Router sends requests to frontier models, open models or models trained for the customer, and it offers custom training on that data⁠18. The limits are plain: it is not a model, it has no published case studies of a custom-trained model yet, and custom training is priced per engagement.

What success looks like

An inventory exists and names an owner for every source of company knowledge.
The data is current, with duplicates and conflicts resolved and personal data minimised.
An evaluation set of real tasks measures every model the same way.
A trained model beats the base model on that evaluation set, by a margin that is written down.
It runs behind the router, receiving the work it does best, while frontier models handle the rest.
The pipeline repeats. New tickets, documents and decisions flow into the next training round, so the model improves as the company works.

The end state is a model that answers in the company's own terms, trained on knowledge the company owns, that can be retrained on a newer open base model when one appears without starting the data work again.

Questions buyers ask

How do you train an LLM on company data?

Gather the company's knowledge from mail, documents, tickets, chat and system records; remove duplicates and outdated or conflicting material; structure it into examples of the tasks the model should do; set aside an evaluation set from real work; then fine-tune an open-weight model on the rest, usually with a parameter-efficient method such as LoRA, and test it against the evaluation set before routing work to it.

What is the difference between fine-tuning and RAG?

Retrieval-augmented generation leaves the model unchanged and places relevant company documents in the prompt at the moment of each request. Fine-tuning changes the model's weights so it learns the company's terms, formats and ways of working. Retrieval suits facts that change often; fine-tuning suits behaviour and language that should be consistent. Many companies use both.

How much data do I need to fine-tune a model?

There is no fixed number. Parameter-efficient methods such as LoRA train a small number of extra parameters on top of a base model, so they work with far less data and hardware than training from scratch. What matters more than volume is that the examples are clean, current and representative of the work the model will do, and that a separate evaluation set exists to measure the result.

Can I fine-tune an open-source model for commercial use?

Usually yes. Models under Apache 2.0 or MIT, such as Gemma 4, gpt-oss, Mistral Small 4, DeepSeek V4 and OLMo, allow commercial use and modification. Some flagships use custom licences with naming rules or revenue thresholds, mostly when the model is offered to others as a service. Check the licence of the exact model, not the family.

What hardware do I need to fine-tune an open model?

Less than many expect. The QLoRA method fine-tuned a 65-billion-parameter model on a single 48 GB GPU. Google says Gemma 4 can be fine-tuned on a gaming GPU, and OpenAI documents fine-tuning gpt-oss-120b on a single H100 node. The largest flagships still need multi-GPU clusters.

Is training a model on employee or customer data allowed under the GDPR?

It can be. The European Data Protection Board says legitimate interest can be a lawful basis for developing and deploying a model, subject to a three-step test, and that a model trained on personal data cannot always be considered anonymous. Companies should assess this case by case, minimise personal data in training sets and document the assessment.

Sources

British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hosted. The Register, 3 Nov 2026. Accessed 9 Nov 2026.
How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models: talk notes. Sparse Notes. Accessed 9 Nov 2026.
TRL: Transformer Reinforcement Learning. Hugging Face. Accessed 9 Nov 2026.
PEFT. Hugging Face. Accessed 9 Nov 2026.
Gemma 4: Byte for byte, the most capable open models. Google, 2 Apr 2026. Accessed 9 Nov 2026.
openai/gpt-oss-120b. OpenAI (Hugging Face). Accessed 9 Nov 2026.
mistralai/Mistral-Small-4-119B-2603. Mistral AI (Hugging Face). Accessed 9 Nov 2026.
Models overview. Mistral AI Docs. Accessed 9 Nov 2026.
deepseek-ai/DeepSeek-V4.1-Flash. DeepSeek (Hugging Face). Accessed 9 Nov 2026.
Pricing: serverless inference, dedicated inference and GPU clusters. Together AI. Accessed 9 Nov 2026.
allenai/Olmo-3.1-32B-Think. Ai2 (Hugging Face). Accessed 9 Nov 2026.
QLoRA: Efficient Finetuning of Quantized LLMs. arXiv, 23 May 2023. Accessed 9 Nov 2026.
Online serving (OpenAI-compatible server). vLLM. Accessed 9 Nov 2026.
Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models. European Data Protection Board, 18 Dec 2024. Accessed 9 Nov 2026.
Les fiches pratiques IA. CNIL. Accessed 9 Nov 2026.
Questions and answers on the guidelines for general-purpose AI providers. European Commission. Accessed 9 Nov 2026.
AI Architects. OpenLazarus. Accessed 9 Nov 2026.
Pricing. OpenLazarus. Accessed 9 Nov 2026.

Updated 9 Nov 2026: First published.

Also in: Deutsch · Français · Español · Italiano · Nederlands · Polski