Home › Guides › How I train character LoRAs for Krea 2

How I train character LoRAs for Krea 2

Every setting, every measurement, and the reasoning behind both. This is the exact pipeline behind the characters in the shop — no settings withheld.

Most LoRA guides give you a config and a shrug. This one gives you the config and the alternative it was measured against, because almost every number below started as a different number that changed after timing it or looking at the output side by side.

Every figure here is measured on one machine. Training speeds come out of a real train.log. Image generation times come from the file timestamps of actual batches. Nothing is a manufacturer claim.

The hardware

GPU        NVIDIA GeForce RTX 5090 - 32,607 MiB VRAM, driver 591.86
CPU        Intel Core i9-14900KF
OS         Windows 11, training inside WSL2 (Ubuntu 24.04.4 LTS)
RAM        WSL capped at 24 GB in .wslconfig
Stack      PyTorch 2.11.0+cu128, CUDA 12.8
Trainer    ai-toolkit
Previews   ComfyUI, on the Windows side

Everything runs on one card. There is no second GPU to offload previews onto. Training, captioning and image generation all queue for the same 32 GB, and that constraint shaped more decisions than any quality consideration did.

Train on Raw, generate on Turbo

Krea 2 ships in more than one variant, and picking the wrong one costs you either quality or the ability to train at all. Krea 2 Raw (26 GB) is the variant intended for fine-tuning, so that is what trains. krea2_turbo_mxfp8 renders the previews at roughly half the VRAM of bf16.

LoRAs trained on Raw transfer to Turbo. That is the whole trick, and it is what allows training against the full model while rendering previews at 8 steps.

• Turbo needs cfg 1. Its workflow feeds the sampler a ConditioningZeroOut of the positive prompt as its negative. Fine at cfg 1, where the negative is never evaluated. Push the same workflow to cfg 3.5 and you get strong artefacts.
• You cannot make a Turbo version of a merged checkpoint. Turbo is distilled from Raw, not quantised from it. Merge a LoRA into Raw for a standalone checkpoint and that checkpoint will always need ~30 steps.

The dataset

Across every character dataset on disk: median 48 images, mean 49, range 8–101.

40–80 images is the working range. Below forty, the model starts binding incidental details to the character — a background, a jacket, a particular crop. Past a hundred, the extra images mostly repeat what the earlier ones already said.

Around 200 candidates are generated per character, and the dataset is selected from them — for variety of pose, distance, angle, expression, lighting and setting, not for which images look best.

That generalises: anything constant across the dataset and absent from the captions is what the trigger word ends up meaning. If every image is lit the same way and no caption mentions lighting, the trigger learns the lighting.

Captions

Captioning runs on Qwen3-VL-8B-Instruct locally, in batches of 4, falling back to one-at-a-time if a batch fails. Three rules shape every caption.

Trigger first, always

Captions are written trigger, description. Writing description, trigger does not fire the LoRA reliably. Position genuinely matters.

The traits phrase goes in every caption, in full

Every caption for a character carries the same anatomical description. That is what holds the physique together — reference conditioning is heavily face-biased in every model tested here, so the face transfers on its own while the body drifts back to the base model's prior unless the text insists otherwise.

No magnitude words

"Extremely muscular", "very large", and category nouns produced a physique noticeably bigger than the reference. Describe anatomy precisely instead: well-developed deltoids works, huge shoulders overshoots.

What the captions are really doing

This is the part that is easy to get backwards, and an earlier version of this page explained it badly. The rule is not "hide the thing you want to train". The rule is about what changes between your images and what stays the same:

Here is why. At every training step the model reads the caption, looks at the image, and has to explain the difference between them. Whatever is in the image but not in the caption still has to come from somewhere, so it gets attached to the words that are there. The trigger word is in every single caption, so the trigger is what collects it.

That gives two effects, and they pull in opposite directions:

• Same in every image, and not named → it sticks to the trigger word. This is how the face and the body get learned. This is the effect you want.
• Named in the caption → it sticks to those words instead, and it is pulled away from the trigger word.

You do not write "in a kitchen" to teach the model what a kitchen looks like. It already knows what a kitchen looks like. You write it so that the trigger word does not learn the kitchen.

Then why describe the person at all?

A fair question, and it comes up every time. The answer depends on how you plan to generate later, not on how you train.

If you only type the trigger word when you generate, leave the body description out. Describing the body would spread the person across words you are never going to type. A trigger word on its own, with a varied dataset, trains a good character LoRA, and it is far less work.

If you type the trigger word plus a fixed traits phrase, then put that same phrase in every caption. It is now in the training text and in the generation text, so it works with the trigger instead of against it. That is what this pipeline does, and it is why every model card here ships the traits phrase next to the trigger word. The reason is the one given above: the face transfers on its own, the body drifts back to the base model unless the text keeps insisting.

Both approaches work. The one that does not work is describing the body in the captions and then forgetting to describe it when you generate.

A varied dataset does the same job for free

If the clothes, the rooms and the light really do change from image to image, then the person is the only thing left that stays the same. So the person is the only thing that can attach to the trigger word, and there is nothing for the captions to take away.

Captions start to matter when the variety runs out. Thirty of your fifty images shot in the same room, or the same jacket in half of them, and the trigger word will learn the room and the jacket. Naming them in the captions is the cheap repair. The expensive repair is building the dataset again.

A real example

One real character dataset, 41 images, copied off disk. Only the name is replaced. Every caption has the same shape:

<trigger>, <86 words that never change>, <the part that changes>

The part that never changes is 86 words, and it is identical in all 41 captions, character for character:

<character>, narrow tapered jawline, prominent cheekbones, flat broad
forehead, arched moderately thick eyebrows, almond shaped eyes, light
brown irises, thin eyelids, slightly downturned lower eyelid, moderately
spaced eyes, straight narrow nose, small symmetrical nostrils, small
slightly downturned mouth, thin lips, smooth even skin tone, light brown
hair, long natural length, thick voluminous natural texture, deltoids
broad capped, biceps brachii thick peaked, trapezius broad, rectus
abdominis segmented into six well-defined bellies, bust size large, bust
shape full, waist narrow, hips narrow, freckles absent, moles absent,
scars absent,

The part that changes, in three of those same 41 captions:

... wearing a hooded sweatshirt with the hood down, crouched down on her
    heels, a serious, level expression, in a plain domestic kitchen, a
    single hard light source from one side, waist-up shot, shot from
    slightly above, looking down at her

... wearing a summer dress, mid-stride walking towards the camera, in a
    bedroom with an unmade bed behind her, soft diffused daylight from a
    large window, close-up portrait, head and the top of the shoulders,
    shot from directly behind her

... wearing a bikini, standing relaxed with her arms at her sides, calm,
    eyes half closed against the light, in a stairwell, backlit against a
    bright window, her face in soft fill, full length shot from a short
    distance, the whole figure in frame, shot from slightly below,
    looking up at her

Counted across the 40 captions that were built from the generation prompts — the 41st is the original reference photo, captioned by the VLM afterwards:

No outfit and no setting appears more than three times in forty images. That flatness is the whole point. Nothing in the second half of the caption is common enough to become part of the character, and everything in the first half is common to all of them.

Style LoRAs are the mirror image

Everything above flips over. For a style LoRA, describe the content in full — the subject, the objects, the scene, the colours — and say nothing about how it looks. No "oil painting", no "cel shaded", no "film still". The look is the thing that is the same in every image, so leaving it unnamed is exactly what binds it to the trigger word. Naming it would split the look between the trigger and those words, and then you would need both to fire it.

The ai-toolkit config, in full

A real generated config, copied verbatim:

job: extension
config:
  name: <character>
  process:
  - type: sd_trainer
    device: cuda:0
    trigger_word: <character>
    network:
      type: lora
      linear: 32
      linear_alpha: 32
    save:
      dtype: float16
      save_every: 250
      max_step_saves_to_keep: 6
    datasets:
    - folder_path: <dataset>
      caption_ext: txt
      caption_dropout_rate: 0.05
      shuffle_tokens: false
      cache_latents_to_disk: true
      resolution:
      - 512
      - 1024
    train:
      batch_size: 1
      steps: 2000
      gradient_accumulation_steps: 1
      train_unet: true
      train_text_encoder: false
      content_or_style: content
      gradient_checkpointing: true
      noise_scheduler: flowmatch
      optimizer: adamw8bit
      lr: 0.0002
      dtype: bf16
      ema_config:
        use_ema: true
        ema_decay: 0.99
    model:
      name_or_path: <krea2-raw>
      arch: krea2
      quantize: true
      qtype: qfloat8
    sample:
      sampler: flowmatch
      sample_every: 2001
      width: 1024
      height: 1024
      prompts: []
      guidance_scale: 3.5
      sample_steps: 30

Why each setting is what it is

resolution: [512, 1024] — and specifically not 768

The [512, 768, 1024] in ai-toolkit's example configs is inherited from Flex configurations. It is copied boilerplate, not a Krea 2 recommendation. Training at a resolution the base model never saw spends your step budget on a bucket it has no prior for.

linear: 32, linear_alpha: 32

Rank 16 at 1000 steps gave a likeness worth calling "a good 60%". Rank 32 at 1500 steps was noticeably better on the same dataset and seed. For a photographic character, 32 is where it stops improving enough to justify the cost. Alpha equals rank; no scaling games.

steps: 2000

The same two characters were trained at 2000 and at 2500 steps and compared across five framings with identical seeds. The identity was already resolved at 2000. The extra 500 steps bought nothing, and cost about 24 minutes each.

quantize: true, qtype: qfloat8

This is what makes Krea 2 fit. Blackwell has native FP8 tensor cores, so quantising to qfloat8 costs very little and brings a 26 GB model comfortably inside 32 GB with room for the optimizer, the gradients and the cached latents.

train_text_encoder: false

UNet only. Training the text encoder on a 40–80 image dataset overfits the trigger word to the dataset's phrasing rather than to the person.

shuffle_tokens: false

Deliberate. The captions are ordered — trigger first, then the traits phrase, then the variable content. Shuffling tokens would destroy the position rule above.

sample_every: 2001 — previews disabled during training

The step count is 2000, so an interval of 2001 means the sampler never fires, and the prompt list is empty too. This is worth 30–40 minutes on every run: four preview images at sample_steps: 30 cost about five minutes on a 5090, and sampling at every checkpoint means paying that eight times. Previews are rendered afterwards in ComfyUI at 8 steps instead.

Training times, measured

A real run, caught in progress:

1,948 steps in 1 hour 24 minutes. From the checkpoint timestamps of that same run, 250 steps apart:

1,250 steps in 54 minutes, or 2.59 s/step. The long-run average across many characters is 2.45 s/step, about 82 minutes for a full 2000-step character LoRA. The run above is slightly slower because the machine was in use.

Measured and rejected

This is the section worth reading twice.

Training at 0.25 MP

43 minutes instead of 82 — almost exactly half. Rejected: the likeness degrades visibly. Half the time for a worse product is not a trade worth taking when the product is the likeness.

Fizgig, a Krea 2 specific trainer

A trainer built specifically for Krea 2, advertising dramatically faster training. It would not start under WSL2 at all — CUDA driver error: unknown error down four different install routes, with CUDA working fine in isolated tests. On native Windows it does run, and gives 2.97 s/step at 1024, which is slower than ai-toolkit's 2.45. Its advertised "20 minutes" came from training at 0.25 MP, and its standard Krea 2 preset is rank 32/32 — the same as here.

Generating images in ComfyUI

Model      krea2_turbo_mxfp8.safetensors
Sampler    euler
Scheduler  simple
Steps      8
CFG        1.0
Size       2 MP (9:16 for characters)
LoRA       character LoRA at weight 1.0

Measured throughput, from the file timestamps of two real batches: 11.7 s median over 120 images, and 13.2 s over 70. That is end-to-end per image, including saving; the sampling itself is about 8 seconds.

Startup costs 21 seconds to launch plus 8 to load the models, which is why ComfyUI stays open during a batch. The flip side: closing it frees 18 GB of VRAM — measured, from 11.2 GB free to 29.5. On a single-GPU machine that is the difference between being able to start a training run and not.

Things that cost me time

A LoRA that needs weight 1.5 to apply is a symptom, not a setting. Something upstream is wrong — usually the trigger word, the caption position, or an undertrained rank.

Copy-of-a-copy degrades. Bootstrap a character from a small LoRA, generate a synthetic dataset from it, and the bootstrap's likeness becomes the synthetic set's likeness, which becomes V1's. Make the bootstrap better than it needs to be.

Training on a model's own output cannot expand its priors. Synthetic generation works for characters, because the bootstrap supplies an identity the base model does not have. It does not work for shifting a distribution.

Style LoRAs are limited by subject variety, not image count. A style trained on 49 frames that were all night-time urban scenes failed on cars, animals, landscapes and daytime exteriors. Rank 32 and 86 frames changed nothing.

Checking a filename is not checking identity. A copy step here once compared file size to decide whether to reinstall a LoRA, so a retrained LoRA at the same rank was never copied and every test ran against the old weights. Compare hashes.

← All guides