It seems implementing transformers is the new "hello world" of AI assisted coding.
codekansas 50 minutes ago [-]
This is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
chalkycrimp 38 minutes ago [-]
Good chance I'm not the only one thinking this: share the code
datadrivenangel 1 hours ago [-]
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
risyachka 56 minutes ago [-]
Fpga is software-driven, model can verify its output easily.
Zero surprise it did it
datadrivenangel 1 hours ago [-]
it only cost them $10,000 in Claude API usage. :)
hmate9 1 hours ago [-]
You get that much usage in a monthly $200 subscription.
dvh 29 minutes ago [-]
So who's paying the cost?
exe34 27 minutes ago [-]
Corporate customers
dofm 46 minutes ago [-]
And a kitten.
saejox 29 minutes ago [-]
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
Rendered at 20:20:09 GMT+0000 (Coordinated Universal Time) with Vercel.
It seems implementing transformers is the new "hello world" of AI assisted coding.
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
Zero surprise it did it