Fascinating. We hear that the leaps in AI have been made possible by orders of magnitude increases in compute and data availability, and of course that’s substantially true—but exactly how true? It’s a nice exercise in perspective to see how much or how little modern machine learning methods would have been capable of if you brought them by time machine to the 70’s and optimized them for that environment.
rahen 56 minutes ago [-]
Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.
functional_dev 11 minutes ago [-]
Incredible work! Fitting transformer into 32KB RAM is crazy
For those who read this project and do not know PDP-11 it could be hard to understand that working with these memory limits is difficult.
Here is visual guide for PDP11 architecture - https://vectree.io/c/pdp-11-hardware-architecture
Thanks for this amazing project!
AnimalMuppet 59 minutes ago [-]
Woah. Dude has a running PDP-11/34 in 2026? Personally, I find that more impressive than the program.
rahen 48 minutes ago [-]
That thing is a Tamagochi though, it constantly needs attention, pardon the pun. I did most of the development and tuning on ll-34 for that reason.
Rendered at 14:45:27 GMT+0000 (Coordinated Universal Time) with Vercel.
For those who read this project and do not know PDP-11 it could be hard to understand that working with these memory limits is difficult. Here is visual guide for PDP11 architecture - https://vectree.io/c/pdp-11-hardware-architecture
Thanks for this amazing project!