PyTorch

Models and offerings #

Who they are

This hub is for anyone who writes training code, which in 2026 almost always means PyTorch. The framework came out of Meta’s research lab in early 2017 as a Python-first successor to Torch, won researchers over with eager execution and readable stack traces, and by 2020 had displaced TensorFlow in published papers. In 2022 Meta moved governance to the PyTorch Foundation under the Linux Foundation, with AMD, Google, Microsoft, NVIDIA, Intel and others as members. Meta still contributes the largest share of commits.

PyTorch sits underneath nearly everything else on this site. Hugging Face Transformers dropped its other backends in v5 and is PyTorch only. vLLM, NVIDIA TensorRT-LLM and Apple MLX all import or convert PyTorch models. The main alternative for new research code is JAX, which is worth knowing if you work on TPUs.

Tooling and APIs

Install with pip install torch or the selector on the Get Started page, which builds the right command for your accelerator. PyTorch 2.15.0 (28 October 2026) requires Python 3.11 to 3.15 and no longer ships 3.10 wheels. The default Linux and Windows wheel bundles CUDA 13.2 with CUDA 13.4 wheels alongside, which add experimental support for NVIDIA’s Rubin generation. AMD users get ROCm 7.14 and ROCm 10.0 wheels built on AMD’s TheRock packages, which now pull ROCm in as a pip dependency rather than bundling it. Intel GPUs use the XPU backend. On a Mac, the MPS backend needs macOS 15 or newer and in 2.15 gains FP8 (float8_e4m3fn), a public torch.mps.Stream API and faster decode kernels. Our PyTorch training loop cheat sheet covers the API you will use every day, and tensor shapes and broadcasting covers the errors you will hit first.

The minimal shape of a training step has not changed since 1.0:

import torch

torch.manual_seed(0)
x = torch.randn(256, 4)
y = (x @ torch.tensor([1.0, -2.0, 0.5, 3.0]) + 0.1).unsqueeze(1)

model = torch.nn.Sequential(torch.nn.Linear(4, 16), torch.nn.ReLU(), torch.nn.Linear(16, 1))
optimiser = torch.optim.Adam(model.parameters(), lr=0.02)
loss_fn = torch.nn.MSELoss()

for step in range(300):
    optimiser.zero_grad()
    loss = loss_fn(model(x), y)
    loss.backward()
    optimiser.step()

print(f"torch {torch.__version__}, final mse {loss.item():.4f}")

Expected output is torch 2.14.1, final mse 0.0036 on our pipeline machine, with the version string matching whatever you installed. See the beginner lesson on the training loop for why each line is there.

Everything is BSD-3 licensed and free. The cost is hardware. Pain points, in the order you will meet them: CUDA wheel and driver mismatches, torch.compile recompilation stalls on dynamic shapes, distributed hangs that used to be hard to diagnose (2.15 adds a flight recorder that dumps on a stuck collective, which helps), and the MPS backend lagging CUDA on operator coverage.

Migration notes

From TensorFlow or Keras: the mental model flips from compiled graphs to Python that runs line by line. tf.keras.Model becomes torch.nn.Module with an explicit forward, model.fit becomes a loop you write, and the channels-last default becomes channels-first (N, C, H, W). Keras 3 can run on a PyTorch backend if you want to keep Keras code.

From JAX: jax.jit maps loosely to torch.compile, jax.grad to loss.backward() on tensors with requires_grad, and vmap to torch.func.vmap. State lives on modules rather than being passed through pure functions, which is more convenient and less composable.

Model formats: the safe interchange format is safetensors, which Hugging Face made the default and which PyTorch loads directly. torch.save pickles are convenient but execute code on load, so only load them from sources you trust. Export paths are torch.export for a serialisable graph, ONNX via torch.onnx.export for ONNX Runtime, and ExecuTorch for mobile. The frameworks reference compares these side by side.

Watch for

The 2.x series has changed defaults that silently alter behaviour. In 2.15 the nccl backend name selects the rewritten NCCL implementation. If a multi-node job that worked last month now hangs, TORCH_DIST_USE_NCCL2=0 restores the old path. FSDP2 now separates sharded and unsharded gradient dtypes, which changes numerics when reduce_dtype is left as None. Read the backwards-incompatible section of every release blog.

Old GPUs are falling off. CUDA 13.x cannot target Maxwell, Pascal or Volta, so a GTX 1080 or a V100 needs the separate CUDA 12.6 legacy wheel, and that wheel will not get new features. Python 3.10 is gone from 2.15 and its remaining code paths will be removed.

There is no telemetry in the library itself, but torch.compile caches artefacts under ~/.cache and torch.hub downloads code from GitHub repositories on first use. Pin versions in production: minor releases arrive roughly every two months and torch.compile output can change between them. If you depend on a vendor fork (an NVIDIA or AMD container build), note that its version string may not match the PyPI release and its patches may not be upstream yet.