Fine-tune a small model with LoRA on Apple Silicon in under 15 minutes

What you will build

This project fine-tunes a language model on a laptop and lets you check the result by eye. The base model is SmolLM2-135M (135 M parameters, 8 k context, Apache-2.0, released by Hugging Face in October 2024), small enough that a full forward and backward pass runs in a fraction of a second on an Apple Silicon GPU. The task is to rewrite an ISO date such as 1966-06-08 as 8 June 1966. The base model cannot do it: asked to continue ISO: 1966-06-08\nLong:, it echoes the ISO date back. After 60 training steps with a low-rank adapter it converts 20 unseen dates with no mistakes.

You will generate 400 training pairs, measure the base model, attach a LoRA adapter of rank 8 to the attention query and value projections (0.34 % of the parameters), train it with a plain PyTorch loop that scores only the answer tokens, measure again, save the adapter to a 1.8 MB folder, and reload it on a fresh copy of the base model to prove the saved file is what did the work.

It helps to have read the training loop lesson and the LLM lesson. The fine-tuning topic hub explains where LoRA sits among the alternatives, and the efficient ML hub covers why rank matters.

What you need

• A Mac with Apple Silicon (M1 or later) and 8 GB of memory or more. The model, optimiser state and activations need about 1.5 GB. An Intel Mac, Linux or Windows machine works on CPU with the same code and takes a few times longer.
• Python 3.10 or later with torch 2.x, transformers and peft (pip install torch transformers peft). We tested with PyTorch 2.14.1.
• About 270 MB of disk for the model weights. The first run downloads them from the Hugging Face Hub. After that HF_HUB_OFFLINE=1 works.
• No API keys, no hosted services.

The main block was run from an empty directory with the Field Guide’s pipeline environment on an Apple Silicon Mac mini, once on MPS and once forced to CPU. Timings for both are given below.

Design

What LoRA changes. A linear layer computes y = W x. LoRA keeps W frozen and adds B A x, where A is 576 by 8 and B is 8 by 576. The product B A is a 576 by 576 matrix of rank at most 8, so the update to the layer is constrained to a small subspace. B starts at zero, so at step 0 the model is exactly the base model. alpha / r scales the bypass. With alpha = 16 and r = 8 the scale is 2. SmolLM2-135M has 30 transformer layers with a hidden size of 576, and adapting q_proj and v_proj in each one gives 60 adapted layers. A q_proj adapter has 576 x 8 + 8 x 576 = 9,216 weights. A v_proj adapter is smaller, 6,144, because the model uses grouped-query attention with 3 key-value heads of 64 dimensions, so v_proj outputs 192 values rather than 576. Thirty layers times 15,360 is the 460,800 trainable parameters the code prints.

Why only the answer tokens carry loss. The training text is ISO: 1966-06-08\nLong: 8 June 1966 followed by the end-of-sequence token. The model is scored only on predicting the tokens after Long:. The prompt tokens get the label -100, which cross_entropy ignores, and so does padding. Scoring the prompt would spend capacity learning to predict random dates, which is not the task.

Why a synthetic task. Four hundred dates generated from a seed cost nothing, are free of licensing questions, and have an answer you can verify with a glance. The held-out set uses a different seed, so no test date was seen in training. Exact string match is the metric, which is harsh and honest. The same structure (prompt, answer, exact match on a held-out set) is what you want for a real task before you reach for a fuzzier metric.

Why a plain loop and not Trainer. The loop is 14 lines, every one of which you can see. transformers.Trainer would add gradient accumulation, mixed precision, checkpointing and logging, all of which you want for a real run and none of which you need to understand what is happening. Things to try shows the swap.

Device choice. torch.device("mps" if torch.backends.mps.is_available() else "cpu") picks the Apple GPU when there is one. Everything stays in float32 because MPS support for bfloat16 training is still uneven at the time of writing, and at 135 M parameters float32 is affordable.

Code

Train, compare, save and reload

Save this as lora.py. It is one block so that the before and after measurements share a model and tokeniser.

# lora.py: teach SmolLM2-135M to rewrite ISO dates as British long dates with a LoRA adapter.
import random
import time
from datetime import date, timedelta

import torch
from peft import LoraConfig, PeftModel, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer

SEED = 0
random.seed(SEED)
torch.manual_seed(SEED)
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
MODEL_ID = "HuggingFaceTB/SmolLM2-135M"
ADAPTER_DIR = "date-lora"


# --- 1. A synthetic task you can check by eye: 2024-03-07 -> 7 March 2024 ---
def make_examples(n: int, seed: int) -> list[tuple[str, str]]:
    rng = random.Random(seed)
    start = date(1950, 1, 1)
    out = []
    for _ in range(n):
        d = start + timedelta(days=rng.randrange(365 * 100))
        out.append((d.isoformat(), f"{d.day} {d.strftime('%B')} {d.year}"))
    return out


def prompt_for(iso: str) -> str:
    return f"ISO: {iso}\nLong:"


train_pairs = make_examples(400, seed=1)
test_pairs = make_examples(20, seed=2)
print("example pair:", train_pairs[0])

# --- 2. Load the base model and look at what it does before training ---
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32).to(device)


@torch.no_grad()
def generate(m, iso: str) -> str:
    enc = tokenizer(prompt_for(iso), return_tensors="pt").to(device)
    out = m.generate(**enc, max_new_tokens=12, do_sample=False,
                     pad_token_id=tokenizer.eos_token_id)
    text = tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True)
    return text.split("\n")[0].strip()


def exact_match(m, pairs) -> float:
    m.eval()
    return sum(generate(m, iso) == long for iso, long in pairs) / len(pairs)


show = test_pairs[:3]
print("\nbefore training:")
for iso, long in show:
    print(f"  {iso} -> {generate(model, iso)!r}   (want {long!r})")
print(f"  exact match on 20 held-out dates: {exact_match(model, test_pairs):.2f}")

# --- 3. Wrap the model with a rank-8 LoRA on the attention q and v projections ---
config = LoraConfig(r=8, lora_alpha=16, lora_dropout=0.0,
                    target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM")
model = get_peft_model(model, config)
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
total = sum(p.numel() for p in model.parameters())
print(f"\ntrainable params {trainable:,} of {total:,} ({100 * trainable / total:.2f} %)")


# --- 4. Tokenise so the loss is only on the answer tokens ---
def encode_batch(pairs):
    prompts = [prompt_for(iso) for iso, _ in pairs]
    fulls = [prompt_for(iso) + " " + long + tokenizer.eos_token for iso, long in pairs]
    enc = tokenizer(fulls, return_tensors="pt", padding=True)
    labels = enc["input_ids"].clone()
    for i, p in enumerate(prompts):
        n_prompt = len(tokenizer(p)["input_ids"])
        labels[i, :n_prompt] = -100                 # do not score the prompt
    labels[enc["attention_mask"] == 0] = -100       # do not score padding
    return enc["input_ids"].to(device), enc["attention_mask"].to(device), labels.to(device)


# --- 5. A plain training loop ---
STEPS, BATCH, LR = 60, 8, 1e-3
optimiser = torch.optim.AdamW([p for p in model.parameters() if p.requires_grad], lr=LR)
model.train()
t0 = time.perf_counter()
losses = []
for step in range(1, STEPS + 1):
    batch = random.sample(train_pairs, BATCH)
    input_ids, attention_mask, labels = encode_batch(batch)
    loss = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels).loss
    optimiser.zero_grad()
    loss.backward()
    optimiser.step()
    losses.append(loss.item())
    if step % 5 == 0:
        print(f"step {step:3d}  loss {sum(losses[-5:]) / 5:.3f}")
print(f"trained {STEPS} steps in {time.perf_counter() - t0:.1f} s on {device}")

# --- 6. After training, then save and reload the adapter on a fresh base model ---
print("\nafter training:")
for iso, long in show:
    print(f"  {iso} -> {generate(model, iso)!r}   (want {long!r})")
print(f"  exact match on 20 held-out dates: {exact_match(model, test_pairs):.2f}")

model.save_pretrained(ADAPTER_DIR)
base = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.float32).to(device)
reloaded = PeftModel.from_pretrained(base, ADAPTER_DIR).to(device)
print(f"\nreloaded adapter from {ADAPTER_DIR}/: "
      f"exact match {exact_match(reloaded, test_pairs):.2f}")

Expected output on the Mac mini using MPS. The whole script took 19 s, of which 6.3 s was the 60 training steps and most of the rest was loading the model twice and running 46 generations. Forced to CPU on the same machine the script took 45 s, with 30.5 s of training, and printed the same before and after results.

example pair: ('1974-02-09', '9 February 1974')

before training:
  1960-02-24 -> '1960-02-24'   (want '24 February 1960')
  1966-06-08 -> '1966-06-08'   (want '8 June 1966')
  1965-03-25 -> '1965-03-25'   (want '25 March 1965')
  exact match on 20 held-out dates: 0.00

trainable params 460,800 of 134,975,808 (0.34 %)
step   5  loss 1.503
step  10  loss 0.487
step  15  loss 0.076
step  20  loss 0.016
step  25  loss 0.010
step  30  loss 0.002
step  35  loss 0.006
step  40  loss 0.004
step  45  loss 0.002
step  50  loss 0.003
step  55  loss 0.001
step  60  loss 0.001
trained 60 steps in 6.3 s on mps

after training:
  1960-02-24 -> '24 February 1960'   (want '24 February 1960')
  1966-06-08 -> '8 June 1966'   (want '8 June 1966')
  1965-03-25 -> '25 March 1965'   (want '25 March 1965')
  exact match on 20 held-out dates: 1.00

reloaded adapter from date-lora/: exact match 1.00

You will also see a FutureWarning about is_gradient_checkpointing from transformers and a UserWarning from peft about not finding a config file in the base model. Both are library internals at the versions we used and can be ignored.

The loss is per answer token. It starts near 1.5 because the base model already knows that something date-like follows Long: and gets some digits right, and it is under 0.1 by step 15. The task is small and completely regular, so the adapter learns it in a few hundred examples. Real tasks have a longer tail, and the curve flattens well above zero. The point of the exercise is the shape of the loop and the before-and-after comparison, not the final loss.

Use the adapter from another program

Save this as infer.py. It loads the base model, attaches the adapter from the folder and converts one date from the command line. The adapter folder holds adapter_config.json, which records the rank, alpha and target modules, and adapter_model.safetensors, the 1.8 MB of trained weights.

# snippet
# infer.py: load the base model plus the saved adapter and convert one date.
import sys
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-135M")
base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-135M", dtype=torch.float32)
model = PeftModel.from_pretrained(base, "date-lora").to(device).eval()

iso = sys.argv[1] if len(sys.argv) > 1 else "2026-11-13"
enc = tokenizer(f"ISO: {iso}\nLong:", return_tensors="pt").to(device)
with torch.no_grad():
    out = model.generate(**enc, max_new_tokens=12, do_sample=False,
                         pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True).split("\n")[0].strip())

Run

pip install torch transformers peft
python lora.py
python infer.py 2026-11-13

The first lora.py run downloads the model. Set HF_HUB_OFFLINE=1 afterwards if you want to be sure nothing is fetched. On a Mac, Activity Monitor’s GPU history shows the 6 s of training as a short burst.

Test

lora.py prints its own test: exact match on 20 held-out dates goes from 0.00 to 1.00, and the reloaded adapter scores the same as the trained one. If the reloaded score is lower than the trained score, the save and reload path is wrong (the usual cause is calling save_pretrained on the base model instead of the PEFT wrapper).

infer.py 2026-11-13 printed 13 November 2026 in 5.6 s on the Mac mini, most of it loading the model. A date outside the training range, such as 2099-12-31, is worth trying. The training data runs from 1950 to 2049, and the model has never seen a year starting with 20 and ending above 49.

Run python lora.py a second time and the numbers should match the first run. Our MPS run and our CPU run printed identical losses to three decimal places. Not every MPS kernel is deterministic, so small differences in later decimal places are possible on other machines, but the exact-match scores should still be 0.00 and 1.00.

Things to try

• Set r=1. At rank 1 the adapter has 57,600 parameters, an eighth of the rank-8 version. Does it still reach 1.00? Then try target_modules=["v_proj"] only. Finding the smallest adapter that solves the task is the cheapest experiment in fine-tuning and the most informative.
• Reverse the task (long form in, ISO out) by swapping the two sides of each pair. The model has to learn month names to numbers, which is a lookup rather than a format shuffle.
• Add a second task to the same training set, for example Weekday: with the day name from d.strftime("%A"). One adapter can hold both. Watch whether exact match on the first task drops.
• Replace the loop with transformers.Trainer and a DataCollatorForSeq2Seq that pads labels with -100. Confirm it reaches the same loss by step 60 and note what else it logs.
• Train on the 20 held-out dates as well and see that the final score tells you nothing. Then put them back. A held-out set is the only reason the 1.00 above means anything.
• Merge the adapter into the base weights with reloaded.merge_and_unload() and save the result as a plain model. The merged model is the same size as the base model and needs no peft at inference. Compare generation speed with and without the merge.
• Swap in HuggingFaceTB/SmolLM2-135M-Instruct as the base. It has been trained to follow instructions, so the before-training output changes, and so may the number of steps needed.

Summary

• LoRA trains two small matrices per adapted layer and leaves the base weights alone. Here that is 0.34 % of the parameters and a 1.8 MB file.
• Mask the prompt and padding with -100 so the loss measures only what you want the model to produce.
• Generate a task you can verify by eye, keep a held-out set from a different seed, and measure before as well as after.
• Save the adapter, reload it on a fresh base model, and check the score matches. That is the test that the file on disk is the thing you trained.
• A plain PyTorch loop on MPS trains a 135 M model for 60 steps in about 6 s. CPU takes about five times longer and gives the same answer.