Is this just a theory about an architecture or are there actually some benchmarks/results to substantiate it?
jal278 39 minutes ago [-]
appears it's just a theory, sort of surprised this lifted up so far on hn
fc417fc802 2 minutes ago [-]
A fairly obvious theory at that, unless I've critically misunderstood what's being described. As with so many obvious ideas I've always assumed that the reason I haven't come across it is because it doesn't work.
> Realized reasoning gains, hardware efficiency, and RL scaling remain to be established.
Yeah so the first entry in that list is - if I may be so bold - typically what you'd start with at a small scale _before_ writing up and publishing your "genius" idea. This is the usual crank with delusions of grandeur presenting something straightforward that he hasn't tested as though it were a tested and working breakthrough.
jal278 44 minutes ago [-]
not sure w/o context why this is important -- no results/implementation & i believe there are prior combinations of transformers/RNNs. but perhaps I'm missing the relevance/insight
Rendered at 02:40:28 GMT+0000 (Coordinated Universal Time) with Vercel.
> Realized reasoning gains, hardware efficiency, and RL scaling remain to be established.
Yeah so the first entry in that list is - if I may be so bold - typically what you'd start with at a small scale _before_ writing up and publishing your "genius" idea. This is the usual crank with delusions of grandeur presenting something straightforward that he hasn't tested as though it were a tested and working breakthrough.