Introducing Inkling 2 Mini
Inkling 2 Mini beats Nemotron 3.5 Super VL on Vals Index with roughly one-fifth the parameters.
One thoughtful comment unlocks the ability to create threads.
Inkling 2 Mini beats Nemotron 3.5 Super VL on Vals Index with roughly one-fifth the parameters.
One thoughtful comment unlocks the ability to create threads.
Thinking Machines makes the best US open weights models right now. Inkling 2 Mini is a looped transformer with 26B parameters and 3.4B active that beats Qwen-27B. Lots of interesting details in the technical write up:
The model could follow longer chains of lookups when its layers ran ten times, even though it only trained with up to five passes.
Repeating middle layers beat repeating the last layers.
Unlooped models saved 7.5% on training compute but needed 2.4x as much inference compute vs looped models.
Model was trained on 1 trillion tokens distilled from a teacher.