This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept.
The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.
OutOfHere 5 hours ago [-]
LiteLLM doesn't quite live up to its name. With all those features, there is nothing "lite" about it. It is essential for a project to live up to its name.
Imagine Sqlite adding heavy features from Postgresql, e.g. row-level security.
mpyne 2 hours ago [-]
We run it at my org and it's never been a noticeable resource hog. It's actually the best performer between it, our AI observability stack and the front end.
datadrivenangel 4 hours ago [-]
LiteLLM's problem isn't really features, it's how bloated all the features are, and specifically how AI maximalist and janky their dev practices are.
yujonglee 1 hours ago [-]
thanks for the feedback. genuinely curious what you think we could be doing better, especially around our dev practices. Would love to hear specifics.
OutOfHere 32 minutes ago [-]
Didn't yall have a release of a package that stole user keys or such?
sv123 5 hours ago [-]
But imagine Sqlite not supporting joins or window functions... sure they are useful but look how many LOC it adds! Who is the arbiter of what Lite actually means?
khalic 6 hours ago [-]
I strongly recommend the authors rewrite the readme by hand. It’s kind of a snif test for how much care someone put into this project.
bityard 2 hours ago [-]
I have the same complaint about oh-my-pi's readme. The tone is obnoxious. It's somehow jaded and matter-of-fact at the same time. Like it was written by that one guy at work who never misses a chance to brag about how clever he is.
devinpadron 5 hours ago [-]
Agree. The LLM'isms are offputting.
VCFundedGenYer 5 hours ago [-]
Throwing my support for this. Do not use LLMs to write things humans should write.
0xbadcafebee 4 hours ago [-]
This readme is better than most readmes. However they came to making it, it's clearly working
rexpop 4 hours ago [-]
> Avoid generic tangents.
> Please don't post shallow dismissals
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage.
See: Hacker News Guidelines
khalic 3 hours ago [-]
1. Not a tangent, it’s related to the very first content visible on that link.
2. Not a dismissal, an advise from an expert
3. Not complaining, as stated, giving an advise about the optics of using clear LLM prose on the first paragraph
Barbing 2 hours ago [-]
Tangential annoyance?
ravenstine 5 hours ago [-]
Really? I mean, yeah, it's probably written by an LLM, but it's hardly the worst that I've seen. Looks way more straight forward than the modern README featuring a ton of badges, emojis, confusing out-of-context screenshots, "trust me bro" installation instructions, vague elevator pitches, "used by netflix, nasa, disney, good morning america, alex jones, the church of scientology", and other verbiage to create the illusion that the author won't immediately get bored and abandon their glorified dissertation piece. They all scream "give me your github stars" whereas this one doesn't. But I still get what you mean when it comes to the particular 'isms.
4 hours ago [-]
clickety_clack 5 hours ago [-]
One of the 2 dependencies, httpx, isn't really maintained anymore. Pydantic picked it up as httpx2: https://pydantic.dev/docs/httpx2
9dev 3 hours ago [-]
Funny, everything you pruned away is the reason I’m deploying LiteLLM in our platform. Having a reliable way to track token spend per customer across different services is important to us, and LiteLLM handles this well
dlojudice 2 hours ago [-]
It would be great if there were a plugin/extension infrastructure. For example, to write the cache and costs however and whenever I see fit
freshtake 5 hours ago [-]
First off, cool project! It's always great to see derivatives that question the efficiency of the established product.
I think the main thing the readme is missing is the core benefits. Reducing LOC and dependencies is cool, but it would be great to understand if this provides some additional benefits like lower latency or memory requirements.
hopfenspergerj 3 hours ago [-]
I imagine many people code their own LLM client after getting fed up with the bad options out there. It’s very easy with ai coding tools.
This is a 30 minute project with a frontier LLM. I don’t see why anyone would use anyone else’s router. Techniques are valuable today. Libraries are not.
gcgbarbosa 4 hours ago [-]
Actually not. There are so many edge cases. Also these routers are only useful if they have a minimal layer of observability.
Yes, LLMs can do a great job at writing semi-working MVP. Turning it into a usable project still requires a team.
Yeah, maybe for your toy project you can use a LLM written tool.
Also, I am not saying LiteLLM is good either.
arjie 3 hours ago [-]
There are always people who need an entire team to produce something like OP repo. Enterprise FizzBuzz is real after all.
DrStartup 2 hours ago [-]
most software like this will be dematerialized, democratized, and demonetized - companies building in the infra band being increasingly disintermediated
TZubiri 57 minutes ago [-]
Drop the Lite, it's cleaner, just use the godamn LLM directly.
LeBit 5 hours ago [-]
How does it compare to Bifrost?
josephh 5 hours ago [-]
I'm always confused by LLM proxies that claim to support tool calling. Even for Bifrost that claims to be doing it, at least when I was checking it out, I found out that while it injects the list of MCP tools that's available on the proxy-side, it doesn't actually make the call on client's behalf, and clients get confused by it (response returns MCP call request whose tool doesn't exist on the client-side).
robertclaus 5 hours ago [-]
LiteLLM is basically Bifrost in the Python ecosystem.
gcgbarbosa 3 hours ago [-]
Yeah, but bifrost seems tighter and claims to use way fewer resources
also if you had resource issues with your production deployment, keen to hear more details
25 minutes ago [-]
testycool 30 minutes ago [-]
I don't have specifics since it's been months, but LiteLLM on a hetzner server, deployed through Coolify was using 1-4 GB (cannot exactly, and dont know why either).
Bifrost feels very light, though.
Will try LiteLLM if it's in Rust.
Would also appreciate a slicker aesthetic. I got used to Bifrost shadcn with 0 border radius.
yujonglee 25 minutes ago [-]
- We recently migrated to shadcn, which also comes with dark mode support.
- You can subscribe to the issue or email me at yujong at berri.ai
I’m happy to provide any support if you’re willing to try out the initial version.
Rendered at 00:15:51 GMT+0000 (Coordinated Universal Time) with Vercel.
The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.
Imagine Sqlite adding heavy features from Postgresql, e.g. row-level security.
> Please don't post shallow dismissals
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage.
See: Hacker News Guidelines
I think the main thing the readme is missing is the core benefits. Reducing LOC and dependencies is cool, but it would be great to understand if this provides some additional benefits like lower latency or memory requirements.
I’m biased but I think mine is coded to a higher standard than litelm. https://github.com/s-banach/langchaint
Yes, LLMs can do a great job at writing semi-working MVP. Turning it into a usable project still requires a team.
Yeah, maybe for your toy project you can use a LLM written tool.
Also, I am not saying LiteLLM is good either.
Bifrost feels very light, though.
Will try LiteLLM if it's in Rust.
Would also appreciate a slicker aesthetic. I got used to Bifrost shadcn with 0 border radius.
I’m happy to provide any support if you’re willing to try out the initial version.