summaryrefslogtreecommitdiff
path: root/configs/boris-2-0929/RECIPE.md
diff options
context:
space:
mode:
authorjdj <jdj@kunix.org>2026-11-18 14:47:33 -0800
committerjdj <jdj@kunix.org>2026-11-18 14:47:33 -0800
commit5d2e4252e695644c34be605b44a14bd80a98a0a6 (patch)
tree311a72eda4459af7547b9cd05c0956eafb235f2d /configs/boris-2-0929/RECIPE.md
downloadboris-2-5d2e4252e695644c34be605b44a14bd80a98a0a6.tar.gz
boris-2-5d2e4252e695644c34be605b44a14bd80a98a0a6.zip
Boris-2 training code
The code, configuration and data filters used to train Boris-2 (144M, 200B tokens, single V100): model, training loop, data preparation, n-gram table, V100 kernels, final stages and the exact run configs.
Diffstat (limited to 'configs/boris-2-0929/RECIPE.md')
-rw-r--r--configs/boris-2-0929/RECIPE.md9
1 files changed, 9 insertions, 0 deletions
diff --git a/configs/boris-2-0929/RECIPE.md b/configs/boris-2-0929/RECIPE.md
new file mode 100644
index 0000000..087ec5f
--- /dev/null
+++ b/configs/boris-2-0929/RECIPE.md
@@ -0,0 +1,9 @@
+# Boris-2 release candidate (2026-11-11)
+Official-style Open SLM Index 34.19 (HellaSwag 50.26, ARC-E 59.81, ARC-C 32.00, PIQA 72.91, ArithMark-3 official 45.10).
+Lineage:
+1. Boris-2-0929 final.pt (200B tokens, WSD 1-sqrt decay, doubled batch from 170B).
+2. "Nudge": 30M tokens, mix climbmix .55 / cosmo2_how .30 / cosmo_how_v1_tf .05 / ultra_math_l3 .07 / metamath_gsm .02 /
+ orca_math .01 (per-doc draw weights), LR 0.1x peak, 1-sqrt to 0, seed 1234 (probe/pilot_train.py --peak 0.1).
+3. 10M tokens of LibreMath v1 only (decontaminated: 13-gram + 8-gram ArithMark/SVAMP/ASDiv bans), LR 0.02x peak,
+ 1-sqrt to 0, fresh optimizer state (probe/pilot_train.py --peak 0.02).
+Selected from ~10 short final-stage candidates by benchmark score -- disclose in the model card.