Parameter-Efficient Fine-Tuning

LoRA Rank
Visualiser

Drag the sliders to see how LoRA decomposes a weight matrix into two thin trainable matrices — and how rank controls the quality / memory tradeoff.

Matrix dimension (d) 1024

25620484096

LoRA rank (r) 8

13264

Layers adapted 16

11632

Weight decomposition

W — frozen

1024 × 1024

A — trained

1024 × 8

B — trained

8 × 1024

Full fine-tune params

—

all layers

LoRA trainable params

—

all layers

Reduction factor

—

— of full params

Training memory — 16 adapted layers

Full fine-tune

—

LoRA only

—