# Training: From Chaos to One Click

Training is where the status quo hurts most — and where a platform pays for itself fastest. Whether you're training from scratch, fine-tuning a foundation model, or applying post-training, k3ld Model collapses a multi-week engineering project into a guided flow.

## Training from scratch

**Status quo — 2–4 weeks, 1–2 ML engineers, $800–$2,000 in GPU**

- Day 1–2: launch a GPU box, install CUDA, fight PyTorch versioning
- Day 3: write the data loader from scratch — OOM and tokenizer-padding bugs
- Day 4: write the training loop — NaN at step 200, gradient clipping
- Day 5: wandb needs a paid account → fall back to `print()` logging
- Week 2: "loss isn't going down" → restart (6 hrs lost each time); "GPU idle all weekend" → forgot to checkpoint → start over Monday; "Which checkpoint was best?" → grep the logs → guess

**k3ld Model — hours, at a fraction of the status-quo GPU bill**

- Pick a preset (GPT-2 124M / 355M, LLaMA-style, custom) — the platform compiles the architecture
- Pick the data from the catalog you already explored
- Platform generates the PyTorch script, uploads it, dispatches to GPU
- Loss, gradient norm, throughput, and LR stream live every 30s
- Checkpoints auto-saved; resume from any step; test inference from the run
- Spot instances (~65% off) with automatic fallback

## Fine-tuning (LoRA)

**Status quo — 3–5 days per run**

PEFT scripts, target-module guessing, retrain loops, tensorboard setup, wandb accounts, "which adapter did we actually ship?" — with no shared checkpoint registry.

**k3ld Model — 30 minutes**

Pick a HuggingFace base (Llama, Mistral, etc.), point at your dataset (or an LLM Format node that turned docs into instruction data), and the platform auto-configures LoRA (rank/alpha) for the model. One click, live metrics, and the adapter attaches to a deployed endpoint without a full redeploy.

## Post-training

**Status quo — 1–2 weeks**

GRPO/DPO implemented from papers and reference code, data-format debugging, reward-model tuning.

**k3ld Model — 30 minutes**

Pre-built presets for SFT, preference tuning (DPO/GRPO), and continued pre-training — go from base → aligned model entirely on k3ld.

## The NN console (model.k3ld)

Design the architecture visually: Data Source → Input Shape → Token Embedding → Transformer blocks → Output Head → Loss → Train. It dispatches to SageMaker with automatic instance recommendation and checks VRAM fit up front so you can't launch a run that will OOM and burn credits.

The block palette includes DeepSeek V3-style blocks (MLA attention), DeepSeek V4-Flash blocks (HCA/CSA compressed sparse attention), and Kimi K3 blocks (KDA + LatentMoE), plus plain MoE and transformer blocks. Scale presets — `v3_tiny` through `v3_full`, `v4_flash`, `kimi_k3/lite/mini` — give proven configs instead of guessed hyperparameters.

The NN Training dashboard shows live loss and validation curves and full training logs stream to the console. If a run dies, resume from the last checkpoint instead of starting over. Billing is per-minute on the exact instance used, with spot ~65% cheaper when capacity is available.