ArcFlare 0.x embedded llama.cpp in-process through node-llama-cpp. 1.0 drives the stock llama-server binary as a child process instead.
It sounds like an implementation detail. It is the whole release.
What it bought
- No npm dependencies. Not fewer — none. Nothing to rebuild when Node updates, no native module to fail at install time on someone's machine.
- ~28 MB resident. The CLI is a supervisor, not a host. The model's allocator lives in the process that owns the model.
- Your llama.cpp. Whatever build you tuned — Vulkan, CUDA, ROCm — is the one that runs. ArcFlare does not ship a worse copy of it.
- Many models.
llama-serverruns in router mode, serving several models from a directory and loading them on demand. - Real context sizing. Not a fixed 2048 default: the model's own maximum, fitted to the VRAM actually free.
The weights are mapped once, by the engine that owns them. ArcFlare finds the models, picks the settings, starts the process and gets out of the way.
A build that ships kernels is not a build that works
Backend selection is the part that surprised us most.
A llama.cpp build containing kernels for your GPU is not the same thing as a build that can use your GPU. A ROCm build can carry gfx1151 kernels and still enumerate zero devices, because the installed driver will not expose them to HIP.
So ArcFlare probes rather than infers. It finds every llama.cpp build on the machine, asks each one what devices it can actually see, and picks the best one that works — reporting the difference honestly when a promising build turns out to see nothing:
arcflare backend
arcflare doctorPointing a harness at it
The server speaks the OpenAI API on 127.0.0.1:11434, so anything that talks to OpenAI talks to your local model. ArcFlare configures the harness for you — OpenCode, Hermes, Codex — with three rules for other people's config files: back up before the first write, prefer the tool's own config command over editing its file by hand, and only touch the keys we own.
Nothing writes an approval setting into anyone's config. Which commands you are willing to wave through is your standing policy, not something a tool should decide on your behalf for every session that follows.