Old news by now, but you might not be aware that llama-server can do multi-model for a while now,
Meaning that you (and by that I mean your AI agent that has read the llama.cpp code) can write an ini file pointing to your models with parameters optimized for the specific model on your specific hardware. (Optimized by you through testing. Not that AI)
Then, any api client can just select a model and the system does the right thing.
It's great software. It just works.
__
You just need to ignore the cargo culting commandline options on social media.
But you should be listening to the devs.
Have you already enabled ngram-mod (or rather just spec-default)? It is practically free.
jwr 6 minutes ago [-]
Caveat: this kind-of-works. There are plenty of subtle bugs around this. For example, it's hard to have a default model, and clients need to select the model explicitly in every API request, which makes it difficult to work with several models and multiple clients.
mrighele 48 minutes ago [-]
> (Optimized by you through testing. Not that AI)
Why not optimized by AI through testing ? Give it a test set to work on and let it loose.
LoganDark 15 minutes ago [-]
AI doesn't necessarily know what feels like a good tradeoff to you. I'm sure it could help guide you though.
karimf 4 hours ago [-]
Not sure why it's on the front page now, but I highly recommend using llama.cpp for running AI model locally vs using other inference framework, unless you have a very specific requirement.
ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements.
walrus01 4 hours ago [-]
At this point the options are llama-server or vLLM if you're serious about running things at your desk in the under 256GB RAM size class (70B, 120B size models). In addition to, of course, 27B to 35B size things. With of course a ton of compile time build customization options for whatever specific hardware platform you want to run either llama or vllm on.
embedding-shape 59 minutes ago [-]
> At this point the options are llama-server or vLLM
Which last time I checked, both use different formats of the weights, the former GGUF while the latter .safetensors. I mostly end up using vLLM these days and I'm a bit more performance sensitive than what I used to be. Just a shame it's a hassle to share the weights between them with conversion and what not, either batched or on-startup.
markasoftware 3 hours ago [-]
possibly hitting front page because this website is fairly new? For me, it's certainly the first time I've seen a one-liner curl|bash installer for llama.cpp, which was basically the only reason to use ollama.
itake 3 hours ago [-]
does your comment depend on the OS? I thought MLX has better performance on MacOS than llama.cpp
quantumleaper 3 hours ago [-]
The gap was MUCH larger in the past, but in my tests, oMLX and llama.cpp are now very similar (within 10%) in both prompt processing and generation speed. GGUF ecosystem provides a better selection of quants, in my experience Unsloth ones are excellent.
MrScruff 2 hours ago [-]
I thought the main advantage of oMLX is it's less likely to invalidate the KV cache when working with coding agents, which is key when working on a Mac because of the slower prompt processing.
TekMol 19 minutes ago [-]
I tried
curl -LsSf https://llama.app/install.sh | sh
and then
llama serve -hf unsloth/Qwen3-4B-GGUF:Q4_0
Then I get:
W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
Terminated
And the web interface says
Server unavailable
Maybe it gets killed by the OS because it uses too much RAM?
llama.cpp works pretty well for me on the Framework 13 laptop, but the current era of "move fast, break things, rarely fix" (sorry, that's how it feels), bites here quite a bit.
Two examples:
- https://github.com/ggml-org/llama.cpp/pull/25863 Someone's few lines change broke the native (ROCm) support for the AMD GPU inside Framework (and other integrated systems), and any rollback or proper fix is pending for almost a month. Fortunately there's workaround (switching to Vulkan rather than ROCm devices), but both the way the bug was introduced and the way it is not fixed just doesn't give much confidencen
- LM Studio is using llama.cpp internally for GGUF, they ship their own build with their closed source system as "runtimes". Their ROCm runtime does not enable the the AMD GPU inside the Framework, even thought the llama.cpp version would support it. So their runtime keeps telling me that there's no supported AMD GPU -- again, the solution is to use the GPU with the Vulkan devices. Not fixed since Jan at least https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1...
I guess overall it's the worst runtime I've seen so far, except for all the other runtimes out there... I'm a fan, though in some cases I don't have enough knowledge, or I don't have access to fix things, and that feels like a bummer...
bjackman 34 minutes ago [-]
I think ROCm is just a total second class citizen in the space TBH.
It's a shame coz it's not even really what we want, we would obviously all be better served if we could use Vulkan or something. But I guess it's inevitable that a generic framework lags behind here.
If I was AMD I'd hire a whole ecosystem team to sit next to the ROCm people and just support big users like llama.cpp to work better on their HW, e.g. giving OSS maintainers access to their board farms. Maybe they have already done that, in which case I guess I should say I'd double the size of that team.
d3Xt3r 3 hours ago [-]
So are there any alternatives which do actually work well with ROCm OOTB?
imrehg 3 hours ago [-]
I just switched to Vulkan, and be done with it. :)
As much as I can tell, the ROCm version of llama.cpp would be a bit faster on prompt processing, but about the same on the token generation as Vulkan. Real life benchmarks don't seem to give any "ROCm or nothing" sort of vibes. And the difference between the performance of different models are way bigger than the difference between the llama.cpp versions (and versus different runtimes like the llama.cpp/GGUF and the MLX runtimes on Mac for the same models)...
I've tinkered enough with the serving, that I'd rather do something with them with, say 10% slower speed, than spending hours on seting things up again... YMMV
etdznots 3 hours ago [-]
The first one multiple contributors highlighted the PR as urgent andits had lots of review but it appears to be waiting for another review and/or someone that owns the affected hardware to test that the PR fixes the issue, it wpuld be easy for you to test and report whether or not it does, and the second thing is not related to llama.cpp at all
Yes ideally there would be testing every hardware + software combo but this costs engineering time and $$$ money, and you are running on master branch, no master branch of any software is stable, inherently, if you run into issues, just stick to the old hash where stuff worked, why are you insistent on both being at the bleeding edge and experience 0 breakage!
imrehg 3 hours ago [-]
I did report my test results on the first one. :)
The second I didn't say it's any of llama.cpp's "fault", but it is _related_ to llama.cpp since it's being shipped in another system, aye?
Can't stick to the old hash either, because older version have different bugs. E.g. on older versions the same Qwen3.6 model reliably fails to call specific tools due to template issues, while just having the newer llama.cpp version has that fixed. So different versions - different bugs, rather than no bugs.
Why the beating you are trying to gimme, mate? :)
tosh 4 hours ago [-]
I was a bit suspicious of the url but it is also listed on llama.cpp github
(I still deeply distrust curlpipes in general though.)
tosh 4 hours ago [-]
ty for digging this up!
pplonski86 3 hours ago [-]
Yesterday I installed llama.cpp to test it with local AI Data Analyst that I'm building. I was also testing other open LLM providers: Ollama, Jan, vLLM, LM Studio. I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth, contrary to vLLM which required me to reinstall CUDA drivers because by default it installed the latest one. I'm curious if there is a speed difference between the same open LLM model served with different runners.
chii 3 hours ago [-]
> I had older NVIDIA card (RTX 3070) and llama.cpp instalation was smooth
what model was it that you were able to run with the rtx 3070?
pplonski86 1 hours ago [-]
I was able to fit only small models Qwen3.5-4B in RTX3070 which is not very useful for Python and SQL generation thought. When I wan to test larger open LLM models I often just use cloud resources.
car 50 minutes ago [-]
This MacOS app used to be called LlamaBarn. Really excellent to see the fast progress being made.
Official repo, also has documentation how to configure server parameters:
Small tip, install llama.cpp with brew before llama.app, which will pick up the existing llama.cpp. That way it's easier to stay up to date with llama.cpp, since llama.app is on a slower release cadence.
Also, models installed with the hugging face CLI (hf) are picked up by llama.app automatically. The CLI will keep the model cache updated, e.g. when models get updated.
Llama.cpp became part of Huggingface recently.
walrus01 4 hours ago [-]
Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...).
>Anything that suggests curl into bash just plain sketches me out.
How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.
thebeardisred 3 hours ago [-]
Package management provides cryptographic attestation over the entire process, including the scripts run. Nothing is arbitrary nor mutable in the default state. The files touched will be in a predictable place and the scope of privilege escalation is clearly defined.
BloondAndDoom 3 hours ago [-]
But he’s not using a package, he’s downloading from gir repo and build which exactly same getting a bash script from the same repo and executing it.
rusk 3 hours ago [-]
It’s not. Try pulling updates after a rebase. Very much not the same as piping arbitrary disembodied text into your shell.
thecopy 2 hours ago [-]
Except the first time you do it
thecopy 3 hours ago [-]
>Nothing is arbitrary nor mutable in the default state.
What do you mean with this?
JetSetIlly 3 hours ago [-]
It's more about installation location for me. For these types of projects, I don't even install it. I just clone, build and run from there. I have no idea where the bash script is going to put things.
mr_mitm 2 hours ago [-]
Which also makes it not obvious how to uninstall it. pip/npm/cargo etc have well known mechanisms for that.
curl|sh is convenient for container images I guess.
walrus01 4 hours ago [-]
I mean it's probably not, I just haven't got used to it yet. It's about the same level of security as installing a windows app on win2000 25 years ago and blindly downloading a .exe off the internet and running it to get into the install wizard. But indeed I also kind of blindly trust that whatever I'm getting from the debian trixie officially gpg-signed packages isn't backdoored.
One thing I do not do as a matter of practice is install things with a ridiculous number of recursive npm dependencies.
uecker 2 hours ago [-]
I do not blindly trust anything, and come to the conclusion that downloading binaries from Debian trixie is a lot more safe. There is a world of difference between "curl | sh" and downloading from a curated package repository maintained by a respected community with well-defined processes.
danielrmay 3 hours ago [-]
Rendered text cannot be assumed to equal the underlying text, unfortunately
thecopy 3 hours ago [-]
How so? As i understand your point, this would mean we cannot trust GitHub enough to return the same content in git clone vs curl?
danielrmay 3 hours ago [-]
As an example, webfonts can make rendered text differ from the underlying text that ends up on your clipboard.
thecopy 2 hours ago [-]
Sure, but doesn't this assume that you cannot the publisher anyway? So why would you not trust their homepage but trust their source-code
danielrmay 2 hours ago [-]
[dead]
cybrox 3 hours ago [-]
Download and inspect it.
epihelix 3 hours ago [-]
There are also prebuilt binary archives for just about any distribution and inference backend for the latest github release:
You're not wrong about the appearance of sketchiness of that, but.... dude, come on. "It's not hard" is only true because you already know how to do it.
digital_voodoo 2 hours ago [-]
Exactly. That way of assuming everyone is a seasoned dev or sysadmin always baffles me. There are a ton of tinkerers out there, we are legion, and we are often reluctant to ask, especially here (as suggested by another parallel comment) because we don't want to receive an answer like this. Full circle.
tyre 3 hours ago [-]
If you’re unsure how to do it, you can ask others for help or Claude will tell you.
Point Claude Code at a repository and ask how to install it safely. You don’t have to know about make or cryptography of HTTPS or anything, really. It will walk you through the options and risk.
If you have questions about any part of it—i.e. you don’t recognize an acronym or deeply understand why something works—you can ask.
Or ask here! HN is filled with smart humans.
walrus01 3 hours ago [-]
It's literally three steps, assuming you have the equivalent of the debian "build-essential" dependencies installed on your system for cmake, compiler. It's the exact three rows of my post, pasted one at a time into the CLI. The llama-server build guide page is actually pretty good.
I mean, sure, if there's people who can't figure that out, they're probably better off using a GUI that is a wrapper on top of somebody else's precompiled llama-server, like unsloth studio or lm studio. There's a good sized market for that and I wish them well.
ur-whale 4 hours ago [-]
> Anything that suggests curl into bash just plain sketches me out.
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on your Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
walrus01 4 hours ago [-]
One way I prevent possible catastrophic fuckups is that the 'doing code work' box that runs opencode or pi or whatever, is its entirely own separate VM and desktop environment (running as a xen or kvm guest and with its own LVM logical volume as boot/root and /home disk), than the machine running llama-server itself.
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
jakkos 4 hours ago [-]
> security concerns
Yeah I recently tried the coding harness that's recommended here, Pi, in a bubble wrap sandbox and was horrified to learn that it spams multiple warnings at you if you don't give it write access to its own config/extension folder... Everyone else is rawdogging it I guess.
etdznots 3 hours ago [-]
I think the best course is a docker container or a VM, i agree with the sentiment but trying to restrict the permissions of these things is fighting an uphill battle, it’s probably best to let them reign king in a disposable and isolated environment
progval 3 hours ago [-]
It's okay-ish for Rust because you only need the curl|bash for Rustup, which is meant as a development environment. On production you can, for example, use APT to download it from Debian.
uecker 2 hours ago [-]
Backdoored developers are ok?
kelvinjps10 2 hours ago [-]
it's also on the arch repos
topiolli 3 hours ago [-]
What would you win? Cmake is capable of executing shell commands and you end up with a binary that will execute on your machine anyway. At the end of the day it is just a matter of trust anyway, isn't it? I personally use bwrap if I'm not confident about the source.
Cloning a repo and building it is not _that_ hard, but easy installation is often the thing that makes or breaks a product. I believe Ollama proves that point in this context.
etdznots 3 hours ago [-]
Mostly agree but at least with git clone you have a hash and the malicious code has to be served to everyone, and GitHub is less likely to get hijacked by a malicious third party.
But yes, still trusting the project with arbitrary code execution on your machine, including build formulas that pull stuff from the internet and suffer from all the above anyways
prologic 3 hours ago [-]
Is llama.cpp (and thus llama.app) really that much better than Ollama? I've Only ever played with Ollama, so geniously curious to hear other's real-world experiences.
llama.app is just an URL (for the "advertisement" webpages of llama.cpp outside GitHub).
> that much better than Ollama
llama.cpp is the real thing, ollama was a fork that remained inferior.
hhh 3 hours ago [-]
ollama uses llama.cpp
nexawave-ai 4 hours ago [-]
I think I can probably run Gemma 3 12B on my macbook M3 pro with 18GB. The question is, should I do it? This small model is probably not capable of doing a lot or advanced coding or reasoning. What else could it be used for, since it can run locally and privately?
kennywinker 2 hours ago [-]
A quantized Qwen3.6-35B-a3b can run in a similar footprint to gemma 12b, but is smarter. It can do coding tasks, if you specify them at a finer-grain than with bigger models.
dlcarrier 4 hours ago [-]
I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
madushan1000 4 hours ago [-]
Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
walrus01 4 hours ago [-]
the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
gnull 4 hours ago [-]
It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want.
I had a great experience with llama-cpp with Nvidia backend on NixOS.
(Sorry for being that guy.)
numpad0 3 hours ago [-]
I don't know how it works, but does Vulkan not work on Arc?
cptskippy 2 hours ago [-]
Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70.
It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.
OpenArcBob 2 hours ago [-]
[dead]
halyconWays 3 hours ago [-]
llama.cpp is like the ffmepg of AI, and one of the reasons I so greatly dislike ollama is that the latter completely obfuscates that they're a rebrand of the former. Georgi Gerganov and team did all the hard work; ollama is langchain-like VC-bait with a HF download wrapper.
woadwarrior01 2 hours ago [-]
ICYMI, llama.cpp was also VC funded. Search for "ggml" on this page:
no, if everyone knew about llama.cpp, ollama would be done already
Serveurperso 2 hours ago [-]
[flagged]
whateveracct 4 hours ago [-]
[flagged]
mojo-10 3 hours ago [-]
[flagged]
4 hours ago [-]
helsinkiandrew 4 hours ago [-]
I'm confused, is this from Meta? There's no attribution anywhere. Surely releasing an AI tool called llama breaks their trademark if not
reverius42 3 hours ago [-]
It's from https://github.com/ggml-org/llama.cpp -- not associated with Meta, it's been around for years, and surely they know about it -- so I would guess either it's not a trademark violation or they don't care.
HelloUsername 58 minutes ago [-]
> not associated with Meta, it's been around for year
It seems that llama.app is a direct competitor to ollama.com
I can understand the desire for the llama.cpp project to want to own the end user relationship, it is true that previous to this they were a tool provider and not really owning the end user experience.
fmajid 3 hours ago [-]
And ollama were sketchy about not providing proper credit to llama.cpp, even though that’s all they are, a wrapper for it.
ryan_glass 3 hours ago [-]
Ollama uses the llama.cpp backend for inference. I find Ollama noticably slower. Llama.cpp has had a built-in webui (used as llama-server) for a long time now so have owned the user experience too.
Rendered at 09:54:56 GMT+0000 (Coordinated Universal Time) with Vercel.
Meaning that you (and by that I mean your AI agent that has read the llama.cpp code) can write an ini file pointing to your models with parameters optimized for the specific model on your specific hardware. (Optimized by you through testing. Not that AI)
Then, any api client can just select a model and the system does the right thing.
It's great software. It just works.
__
You just need to ignore the cargo culting commandline options on social media. But you should be listening to the devs.
Have you already enabled ngram-mod (or rather just spec-default)? It is practically free.
Why not optimized by AI through testing ? Give it a test set to work on and let it loose.
ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements.
Which last time I checked, both use different formats of the weights, the former GGUF while the latter .safetensors. I mostly end up using vLLM these days and I'm a bit more performance sensitive than what I used to be. Just a shame it's a hassle to share the weights between them with conversion and what not, either batched or on-startup.
When I try
It seems to work. Nice.Two examples:
- https://github.com/ggml-org/llama.cpp/pull/25863 Someone's few lines change broke the native (ROCm) support for the AMD GPU inside Framework (and other integrated systems), and any rollback or proper fix is pending for almost a month. Fortunately there's workaround (switching to Vulkan rather than ROCm devices), but both the way the bug was introduced and the way it is not fixed just doesn't give much confidencen
- LM Studio is using llama.cpp internally for GGUF, they ship their own build with their closed source system as "runtimes". Their ROCm runtime does not enable the the AMD GPU inside the Framework, even thought the llama.cpp version would support it. So their runtime keeps telling me that there's no supported AMD GPU -- again, the solution is to use the GPU with the Vulkan devices. Not fixed since Jan at least https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1...
I guess overall it's the worst runtime I've seen so far, except for all the other runtimes out there... I'm a fan, though in some cases I don't have enough knowledge, or I don't have access to fix things, and that feels like a bummer...
It's a shame coz it's not even really what we want, we would obviously all be better served if we could use Vulkan or something. But I guess it's inevitable that a generic framework lags behind here.
If I was AMD I'd hire a whole ecosystem team to sit next to the ROCm people and just support big users like llama.cpp to work better on their HW, e.g. giving OSS maintainers access to their board farms. Maybe they have already done that, in which case I guess I should say I'd double the size of that team.
As much as I can tell, the ROCm version of llama.cpp would be a bit faster on prompt processing, but about the same on the token generation as Vulkan. Real life benchmarks don't seem to give any "ROCm or nothing" sort of vibes. And the difference between the performance of different models are way bigger than the difference between the llama.cpp versions (and versus different runtimes like the llama.cpp/GGUF and the MLX runtimes on Mac for the same models)...
I've tinkered enough with the serving, that I'd rather do something with them with, say 10% slower speed, than spending hours on seting things up again... YMMV
Yes ideally there would be testing every hardware + software combo but this costs engineering time and $$$ money, and you are running on master branch, no master branch of any software is stable, inherently, if you run into issues, just stick to the old hash where stuff worked, why are you insistent on both being at the bleeding edge and experience 0 breakage!
The second I didn't say it's any of llama.cpp's "fault", but it is _related_ to llama.cpp since it's being shipped in another system, aye?
Can't stick to the old hash either, because older version have different bugs. E.g. on older versions the same Qwen3.6 model reliably fails to call specific tools due to template issues, while just having the newer llama.cpp version has that fixed. So different versions - different bugs, rather than no bugs.
Why the beating you are trying to gimme, mate? :)
https://github.com/ggml-org/llama.cpp
(I still deeply distrust curlpipes in general though.)
what model was it that you were able to run with the rtx 3070?
Official repo, also has documentation how to configure server parameters:
https://github.com/ggml-org/Llama-macOS
Small tip, install llama.cpp with brew before llama.app, which will pick up the existing llama.cpp. That way it's easier to stay up to date with llama.cpp, since llama.app is on a slower release cadence.
Also, models installed with the hugging face CLI (hf) are picked up by llama.app automatically. The CLI will keep the model cache updated, e.g. when models get updated.
Llama.cpp became part of Huggingface recently.
Git clone llama.cpp and build it, it's not hard.
https://github.com/ggml-org/llama.cpp/blob/master/docs/build...
literally just a few steps for the basics:
git clone https://github.com/ggml-org/llama.cpp
cmake -B build
cmake --build build --config Release
How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.
What do you mean with this?
curl|sh is convenient for container images I guess.
One thing I do not do as a matter of practice is install things with a ridiculous number of recursive npm dependencies.
https://github.com/ggml-org/llama.cpp/releases
No need to compile unless you really need to.
Point Claude Code at a repository and ask how to install it safely. You don’t have to know about make or cryptography of HTTPS or anything, really. It will walk you through the options and risk.
If you have questions about any part of it—i.e. you don’t recognize an acronym or deeply understand why something works—you can ask.
Or ask here! HN is filled with smart humans.
I mean, sure, if there's people who can't figure that out, they're probably better off using a GUI that is a wrapper on top of somebody else's precompiled llama-server, like unsloth studio or lm studio. There's a good sized market for that and I wish them well.
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on your Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
Yeah I recently tried the coding harness that's recommended here, Pi, in a bubble wrap sandbox and was horrified to learn that it spams multiple warnings at you if you don't give it write access to its own config/extension folder... Everyone else is rawdogging it I guess.
Cloning a repo and building it is not _that_ hard, but easy installation is often the thing that makes or breaks a product. I believe Ollama proves that point in this context.
But yes, still trusting the project with arbitrary code execution on your machine, including build formulas that pull stuff from the internet and suffer from all the above anyways
llama.app is just an URL (for the "advertisement" webpages of llama.cpp outside GitHub).
> that much better than Ollama
llama.cpp is the real thing, ollama was a fork that remained inferior.
I had a great experience with llama-cpp with Nvidia backend on NixOS.
(Sorry for being that guy.)
https://github.com/cptskippy/battlemage-llm-gateway
It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.
https://aigrant.com
https://huggingface.co/blog/ggml-joins-hf
This post (https://news.ycombinator.com/item?id=35100086) from march 2023 says in the title "Llama.cpp: Port of Facebook's LLaMA model in C/C++"
Must be tough not to be able to monitor your own models!
(The odds that that tagline was AI-generated seem high.)
> Visit https://llama.app and follow the instructions
It's linked at the start of the README.
I can understand the desire for the llama.cpp project to want to own the end user relationship, it is true that previous to this they were a tool provider and not really owning the end user experience.