Same reason the human brain has a train of thought: there's a lot of stuff you "know" as episodic memories of words you've heard/read, but haven't yet internalized via practice into reflex reactions. It's a lot faster and cheaper to acquire a memory of some words, than it is to acquire an entire new skill, so you end up with far more memories-of-words than concrete skills.
Reasoning (i.e. bubbling those remembered words up into your mental context, and then using a special "acting as constrained by words" bridging reflex on them, a.k.a. "system-2 thinking") is how you leverage memories-of-words into useful behavior.
An especially useful version of this is planning, where you chain through "mere words" until you end up at words that declare to yourself what you're about to do; and then you rely on your bridging reflex to react in a way that's constrained by the plan you generated.
dogscatstrees 22 minutes ago [-]
Wang's argument contradicts the view that intelligence necessarily improves through more reasoning and undermines any assumption that recursive self-improvement (RSI) will automatically converge toward greater intelligence.
vova_hn2 11 minutes ago [-]
I'm pretty sure that the current approach of baking in chain-of-thought reasoning into weights is not the end of the line. It just seems to accidental and ad-hoc.
P. S. The animated bird is truly magical!
avereveard 12 minutes ago [-]
Normalizes user voice to goal so the action has more chances to success you can see this on model that dont hide traces sending very badly spelled or incomplete sentences
26 minutes ago [-]
jrflo 17 minutes ago [-]
Content aside, that's a beautiful webiste
akavi 6 minutes ago [-]
My understanding of the math is weak (to say the least), but intuitively what I'm getting from this is:
1) Pretraining creates a bias _towards_ "correct answers" with each inference step
2) A longer walk is more likely to land in the correct region, given it's easier to verify you have the correct answer than to create it
3) RL selects the subset of walks that lands in the correct region and tilts the distribution in favor of such walks
4) By 2, that trains towards longer walks
...Is that correct?
jebarker 28 seconds ago [-]
Yes. One point worth making though is that pre-training trains the model to produce high probability completions - in practice, most of what it's trained on is a good completion as a human would judge it, i.e. it's not factually false data, so you get "creates a bias _towards_ "correct answers" with each inference step", but the model isn't really learning what correct or incorrect means in any way a human would consciously think about it.
NickNaraghi 31 minutes ago [-]
> reasoning as pathfinding through token space
Sounds like it's hillclimbing all the way down!
Cool framing. I'm excited for the parallel construction of ___ as pathfinding through weight space.
FLeXMurphy 51 minutes ago [-]
What is the library, if any, used for the equation/formula rendering? Looks tastefully done.
freecodeio 46 minutes ago [-]
> The mathematics on this page is being rendered by MathJax, which generates both the text spoken by screen readers, as well as the visual layout for sighted users.
FLeXMurphy 44 minutes ago [-]
Lovely. Thank you.
domelian 20 minutes ago [-]
also very curious, visualization is great!
Rendered at 20:50:44 GMT+0000 (Coordinated Universal Time) with Vercel.
Reasoning (i.e. bubbling those remembered words up into your mental context, and then using a special "acting as constrained by words" bridging reflex on them, a.k.a. "system-2 thinking") is how you leverage memories-of-words into useful behavior.
An especially useful version of this is planning, where you chain through "mere words" until you end up at words that declare to yourself what you're about to do; and then you rely on your bridging reflex to react in a way that's constrained by the plan you generated.
P. S. The animated bird is truly magical!
1) Pretraining creates a bias _towards_ "correct answers" with each inference step
2) A longer walk is more likely to land in the correct region, given it's easier to verify you have the correct answer than to create it
3) RL selects the subset of walks that lands in the correct region and tilts the distribution in favor of such walks
4) By 2, that trains towards longer walks
...Is that correct?
Sounds like it's hillclimbing all the way down!
Cool framing. I'm excited for the parallel construction of ___ as pathfinding through weight space.