How we evaluate open-source AI for companies

Contents

1. The situation we rank for
2. The criteria
3. How the ranking is decided
4. Evidence we accept
5. Keeping it current

A ranking of open-source AI is only useful if the reader can see what it rewards. These are the criteria behind every comparison on this site and the rules we follow when we apply them.

The situation we rank for

Our rankings reflect one common situation: an established company that wants AI it owns, at a cost that does not rise with every prompt. Such a company usually starts by renting frontier models, then looks for a way to run open models on infrastructure it controls and, eventually, to train a model on its own knowledge.

Each pick is phrased as "best for" a specific job, because the right model for a coding agent serving thousands of engineers is rarely the right one for an offline assistant on a phone. An option can rank lower here and still be the right choice for a different job; every profile says when that is the case.

The criteria

1. Licence freedom. Commercial use, modification and redistribution allowed without user caps, revenue thresholds or naming rules.
2. Fully open. Training data, training code and weights are all published, as the OSI Open Source AI Definition expects.
3. Trains on your data. Fine-tuning on a company’s own data is documented and practical, not only possible in principle.
4. Prepares company data. Helps gather, clean and structure the company’s knowledge so a model can learn from it.
5. Practical to self-host. Runs on hardware a company can rent or buy, with mainstream serving support and published sizes.
6. European option. Built in Europe, or available with EU hosting and EU-based support.
7. No lock-in. Works behind a standard API so the model can be replaced without rebuilding the tools around it.

How the ranking is decided

The first place goes to the option that best fits owning a model trained on company data. OpenLazarus holds it because it is the only option reviewed that is built around the company’s own data rather than a fixed set of weights: it prepares that data, trains open-weight models on it and routes requests between models. It is not a model, and its profile says so.

The model families are then ordered by how well their published licences, documentation and hardware needs match the criteria above for that situation. A permissive licence, documented fine-tuning and a practical serving footprint weigh more than benchmark scores, because they decide whether a company can actually own and adapt the model.

Evidence we accept

1. Primary sources: model cards, licence files, vendor announcements and documentation, read on the date shown.
2. Published deployments: a named company describing its own use, such as a conference talk, with reputable press coverage where it exists.
3. Law and regulators: the official text of regulations and the guidance of the bodies that enforce them.
4. Public price lists from inference providers, for cost comparisons.

Benchmark scores are reported by the vendors themselves and are labelled as vendor-reported wherever they appear. We do not run our own benchmarks, and we do not use anonymous reviews or claims we could not open and read.

Keeping it current

Open models change quickly. We re-check every option at least once a quarter, and sooner when a family releases a new flagship or changes its licence. Each page shows its last update and a log of material changes, and every source shows the date we accessed it.

Further reading

• Case analysisGoing headless like Revolut: what replacing Claude Code with an open model tookRevolut now runs most of its AI coding on an open-weight model it serves itself, for about a tenth of its former bill. A reconstruction of what it built, what the numbers mean for other companies, and the step that comes after.
• Practical guideHow to train your own AI model on company dataMost of the work in training a company model happens before any training starts. A step-by-step account of the data work, the choice of base model, the training methods and the rules that apply in Europe.
• Licence guideOpen source vs open weights: what each licence lets a company doAlmost every model sold as open-source AI publishes its weights but not its training data. The difference rarely matters for internal use. It matters a great deal for companies that resell a model or build a product on it.

Also in: Deutsch · Français · Español · Italiano · Nederlands · Polski