← Back to Labs

LLM Fine-Tuning & Weight Alignment

Step through SFT loss optimization, LoRA low-rank matrix decomposition W = W0 + (alpha/r)*B*A, QLoRA NF4 quantization, and DPO

Base Weight W0 (d × k)Frozen Base WeightsW0FP16 (16-bit Floating Point)Update PathwayUpdates InhibitedNo Active AdapterW0 Remains StaticAdapter: DisabledTraining & Alignment MetricsLOSS OBJECTIVEN/A (Initialization Baseline)VRAM Memory Footprint14.0 GB (Base Model Load)DPO (Implicit Reward Pair Loss)EWC / KL Penalty (Forgetting Guard)W0 Base Model Frozen
STEP 1 OF 6

Base Model Weight Matrix Initialization

Large Language Models start with frozen pre-trained weight matrices (W0) stored in high-precision FP16 or BF16 formats. For a 7B parameter model, storing W0 alone takes ~14 GB of GPU memory before any training begins.

Arrow keys to navigate · R to reset

Tap dots to jump to any step

Read the full article →Take the quiz →