AI/ML Field Guide
Topic hub

Open-source ML ecosystem

The open-source stack from framework to local runtime, what each layer is for, and the licence distinctions between open source, open weights and restricted weights that decide what you may ship.

Updated 13 Nov 2026 · PyTorch, JAX, Hugging Face, vLLM, llama.cpp, Ollama, open weights, licences

Overview

Nearly every model you will use is trained with open-source software, and a growing share of the models themselves can be downloaded and run on your own hardware. The ecosystem has settled into layers: a framework (PyTorch or JAX), a model library and hub (Hugging Face), serving and local runtimes (vLLM, llama.cpp, Ollama) and the weights that flow through them. Each layer has its own licence, and the weights’ licence is the one that most often decides what a product may do.

This hub is for engineers choosing the stack and reading the terms. It explains what each layer does, which tools to pick, and the difference between open source and open weights, which is not a pedantic distinction: it decides whether you can fine-tune, redistribute, serve commercially, or train another model on the outputs. Frameworks are compared at /reference/frameworks/, and /reference/models/ records each model’s licence.

Key ideas

The stack has four layers

frameworks: PyTorch (BSD-3), JAX (Apache-2.0)tensors, autograd, compilers, CUDA, Metal and TPU backendsmodel code and hub: Transformers, Diffusers (Apache-2.0)architectures, tokenisers, safetensors, model cards, cacheserver runtimevLLM (Apache-2.0)GPU, batching, OpenAI-style APIlocal runtimellama.cpp, Ollama (MIT)GGUF, CPU and Apple Siliconyour applicationbound by every licence below it, and above all by the weightsweightssafetensors or GGUFApache-2.0 / MIT:permissiveLlama, Gemma terms:open, restrictednon-commercial:research onlyclosed: API only,terms of servicethe weights’ licence passes through every layer unchanged
The open-source stack from framework to application. The code at every layer is under a permissive licence. The weights are the exception: their licence travels with them through every layer and binds the application.

PyTorch is the default framework for research and most production training. JAX is a functional array library with the XLA compiler in the loop, favoured on TPUs. Hugging Face’s Transformers and Diffusers libraries provide reference implementations of most published architectures, and the Hub provides the weights, tokenisers and model cards, with safetensors as the file format. vLLM serves models on GPUs behind an API that mimics OpenAI’s, which lets applications switch between hosted and self-hosted models by changing a URL. llama.cpp runs quantised GGUF files on CPUs and Apple Silicon, and Ollama wraps it with a model registry and a one-command local server.

Open source is not the same as open weights

Open source has a definition: the Open Source Initiative’s, which requires that anyone may use, study, modify and redistribute for any purpose, with no field-of-use restrictions. For models, the OSI published an Open Source AI Definition in October 2024 that additionally requires enough information about the training data to recreate a substantially equivalent system, plus the training code and the weights under OSI-approved terms. Very few released models meet it. OLMo 2 (7 B, 13 B and 32 B parameters, 4 k context, Apache-2.0, November 2024 and March 2025) from the Allen Institute releases weights, code, training data and logs, and is the clearest example.

Most of what gets called open source is open weights. The weights are downloadable and the licence may be permissive (Apache-2.0 for Mistral 7B, Qwen2.5 and Qwen3, MIT for DeepSeek-R1), but the training data and often the training code are withheld, so the model cannot be reproduced. That is a fine basis for products and still not open source by the definition. A third group is open weights under custom terms. The Llama 3.1 Community Licence (Llama 3.1 8B, 70 B and 405 B, 128 k context, July 2024) requires attribution, carries an acceptable-use policy, and requires companies above 700 M monthly active users to request a separate licence. The Gemma Terms of Use (Gemma 3, 1 B to 27 B, March 2025) include a prohibited-use policy that Google may update. These are open weights, restricted. Research-only licences such as FLUX.1 [dev]’s, and API-only closed models whose terms commonly forbid training on outputs, complete the table.

Licence type Examples Weights Code and data Commercial use Fine-tune and redistribute Train on outputs
Open source (OSI definition) OLMo 2, Pythia Apache-2.0 Released Yes Yes Yes
Open weights, permissive Mistral 7B, Qwen3, DeepSeek-R1 Apache-2.0 or MIT Data withheld, code partial or absent Yes Yes Yes
Open weights, restricted Llama 3.x, Gemma 3 Custom community licence or terms of use Withheld Yes, with conditions and a use policy Yes, derivatives carry the licence and naming rules Llama 3.1 permits it if the new model carries a Llama name prefix, Gemma terms differ, check each
Responsible AI licences Stable Diffusion 1.5 and SDXL (OpenRAIL-M) Custom with use restrictions Weights only Yes, outside the restricted uses Yes, under the same restrictions Not addressed
Non-commercial or research FLUX.1 [dev], many academic releases Custom Weights only No Research only Usually no
Closed GPT-4o, Claude, Gemini Not available Withheld Via API, under terms of service No Usually forbidden

Reading a licence before you ship

The questions to answer for any model are the same. May you use it commercially, and above what scale. May you fine-tune it and release the result, and under what name. May you use its outputs to train another model. Must you display a notice or carry the licence with the artefact. Record the answer in the model registry (see /topics/mlops-and-deployment/) next to the version, because the licence of a quantised derivative on the Hub is often not the licence of the original.

Where to start

Read /learn/beginner/07-your-first-neural-network/ to meet PyTorch, /learn/beginner/09-using-an-llm-well/ for the API shape that vLLM and Ollama imitate, and /learn/beginner/10-from-notebook-to-service/ for the service layer. Then install Ollama, pull a small model with an Apache-2.0 licence, and point the /projects/tiny-rag-local-embeddings/ project at it. /platforms/opensource/ lists the projects, and /topics/efficient-ml/ explains the GGUF quantisation levels.

Tools and platforms

Tool What for Licence Platform
PyTorch Training and research framework, default for most published code BSD-3 /platforms/pytorch/
JAX and Flax Compiled array programming, TPU-first training Apache-2.0 /platforms/jax/
Transformers, Diffusers, safetensors, Hub Model implementations, weights and model cards Apache-2.0 /platforms/huggingface/
vLLM GPU serving with an OpenAI-compatible API Apache-2.0 /platforms/opensource/
llama.cpp CPU and Apple Silicon inference, GGUF format MIT /platforms/opensource/
Ollama Local model registry and server built on llama.cpp MIT /platforms/opensource/
Llama, Gemma, Mistral weights Open-weights model families, licences differ See table above /platforms/meta/, /platforms/google/, /platforms/mistral/

Common mistakes

  • Calling a Llama or Gemma model open source in documentation or a pitch. They are open weights under restricted terms.
  • Pulling a quantised model from the Hub without checking that the uploader’s licence matches the original maker’s.
  • Loading .bin pickle checkpoints from untrusted uploaders. Use safetensors, which cannot execute code on load.
  • Building on a model with a non-commercial licence, then discovering it at launch.
  • Assuming an OpenAI-compatible endpoint behaves like OpenAI’s. Tool calling, JSON mode and stop sequences differ between servers.

Going deeper