NHacker Next
- new
- past
- show
- ask
- show
- jobs
- submit
login
I didn't see any benchmarks against vllm, sglang, exllama, etc
Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.
Rendered at 21:49:00 GMT+0000 (Coordinated Universal Time) with Vercel.