This is fun and games, but wait until an agents finds this and does the most misaligned thing with it in order to fold paper clips.
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
jazzyjackson 25 minutes ago [-]
> When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
traverseda 30 minutes ago [-]
I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.
horacemorace 21 minutes ago [-]
Along these lines I’m curious how a scratch trained model with a few built-in tools (memory, calculator, ?) available during a single generation would behave. Not tool calls as turns.
low_tech_punk 18 minutes ago [-]
a single generation would still allow looping in the "CPU" because the instruction set can do it. The model is almost a Turing machine (except for the limit on memory)
farhanhubble 37 minutes ago [-]
This is cool and I understand the RAM emulation with gated attention but how does the CPU emulation wor? You add layers that are then trained to predict outputs while interpreting RISC instructions?
bee_rider 23 minutes ago [-]
I think they describe the CPU using the weights? I mean in some sense a CPU can be viewed as just a function, right?
ram_rattle 35 minutes ago [-]
Cool cool cool
rvz 26 minutes ago [-]
So this is yet another rube goldberg machine.
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
If anyone else has an easier way to emulate RISC-V I’d like to hear it. /S
gigel82 30 minutes ago [-]
But... why? </s>
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
27 minutes ago [-]
Rendered at 01:56:59 GMT+0000 (Coordinated Universal Time) with Vercel.
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
Why?
[0] https://arxiv.org/abs/1410.5401
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.