Home › Guides › Baking a LoRA into a checkpoint: what a merge really does

Baking a LoRA into a checkpoint: what a merge really does

Baking a LoRA into a checkpoint sounds like an upgrade. It is not. It is a convenience, and knowing exactly what it does saves you a lot of disappointment.

What a merge is

A LoRA is two small matrices per layer, A and B. Loading it at weight w adds B · A · w to the model's weights while you generate. A merge does the same addition once and saves the result.

If you want a model where everything comes out in a style no matter what you ask, a merge will not do it. That needs a real fine-tune.

When it is worth doing

• One file instead of two. Easier to share, and nobody has to know which LoRA to load.
• Baking a weight above 1. One style LoRA I trained needed weight 1.6 to convert everything. Baked at 1.6, the checkpoint works at the settings everybody uses, with no explanation needed.
• Several LoRAs fixed together, each at its own weight.

How to do it in ComfyUI

It takes minutes, not hours:

UNETLoader  (the base diffusion model)
   ↓
LoraLoaderModelOnly  (your LoRA, at the weight you want to bake)
   ↓
ModelSave

1. ModelSave writes into ComfyUI/output/diffusion_models/.
2. Move the file to ComfyUI/models/diffusion_models/.
3. Load it with a plain UNETLoader, without the LoRA.

On Krea 2 Turbo mxfp8 the result is the same size as the original, 13.5 GB: it keeps the quantisation.

KeyError: 'F8_E8M0'

If you merged on an mxfp8 model, ComfyUI may refuse to load your new checkpoint with:

KeyError: 'F8_E8M0'

The merge itself is fine, and so is the file. mxfp8 models store a scale per block of weights, 256 tensors of them in Krea 2 Turbo. The original file stores those scales as U8. ModelSave writes them as F8_E8M0, which is the correct safetensors type for mxfp8 scales. The problem is that ComfyUI's fast loader (the _TYPES table in comfy/utils.py) knows F8_E4M3, F8_E5M2 and U8, but not F8_E8M0 yet.

Both types are one byte with the same bits. So you do not need to touch the data: relabel those scales as U8, the way the official mxfp8 files store them, and ComfyUI loads the file.

The fix, in 25 lines

A .safetensors file starts with 8 bytes giving the header length, then a JSON header, then the data. The script rewrites only the JSON and pads it with spaces to exactly the old length, so every data offset stays valid. The 13.5 GB of weights are never read or written.

import json, sys

OLD, NEW = "F8_E8M0", "U8"           # swap them to undo

path = sys.argv[1]
with open(path, "r+b") as f:
    size = int.from_bytes(f.read(8), "little")
    header = json.loads(f.read(size))
    changed = [k for k, v in header.items()
               if k != "__metadata__" and v.get("dtype") == OLD]
    for k in changed:
        header[k]["dtype"] = NEW
    new = json.dumps(header, separators=(",", ":")).encode("utf-8")
    if len(new) > size:
        sys.exit("new header does not fit")
    new += b" " * (size - len(new))   # spaces are valid padding
    f.seek(8)
    f.write(new)

print(f"{len(changed)} tensors relabelled {OLD} -> {NEW}")

Run it as python fix_e8m0.py my_merge.safetensors. Make a copy first if the file matters to you.

Or avoid the problem: merge on the plain fp8 version of the model instead of mxfp8. It has no block scales, so there is nothing to relabel. That is what I do now for every Krea 2 merge.

How to check the merge worked

1. Pick a few prompts and seeds. Generate them with the original model + the LoRA at the weight you baked.
2. Generate the same prompts and seeds with the merged checkpoint and no LoRA.
3. They should be the same images.
4. For a stronger check, compare a few tensors of both files. If they are identical, the LoRA never went in.

Use prompts with no style words. My first check of a style merge looked like a failure: the first prompts asked for Van Gogh, ukiyo-e and pixel art, and showed no trace of the baked style. When the prompt names another style, the prompt wins. That says nothing about the merge.

Raw, Turbo and what you cannot do

Krea 2 comes as Raw (26 GB, for training) and Turbo (distilled, 8 steps).

• You can merge into Turbo. The LoRA was trained on Raw, but LoRAs trained on Raw transfer to Turbo, and so do merges.
• You cannot make a Turbo version of a Raw merge. Turbo is distilled from Raw, not compressed from it. A checkpoint merged on Raw will always need about 30 steps.

FAQ

Does merging a LoRA into a checkpoint improve quality?

No. A merge at weight w gives exactly the same images as loading the LoRA at weight w. It only puts everything in one file.

How do I fix KeyError 'F8_E8M0' in ComfyUI?

ModelSave saves the block scales of an mxfp8 model as F8_E8M0, the correct safetensors type, but ComfyUI's loader does not know it yet. Relabel them as U8 in the safetensors header; the data is the same byte.

How long does a LoRA merge take?

A few minutes in ComfyUI with UNETLoader, LoraLoaderModelOnly and ModelSave.

← All guides