{"full_name":"llamastash/llamastash","owner":"llamastash","name":"llamastash","shard":"1","url":"https://github.com/llamastash/llamastash","description":"Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.","homepage":"https://llamastash.dev","topics":["ai","gguf","llamacpp","llm","lmstudio","local-ai","ollama"],"tag_pages":["ai","gguf","llamacpp","llm","lmstudio","local-ai","ollama"],"language":"Rust","license":"MIT","avatar":"https://avatars.githubusercontent.com/u/286277587?v=4","created_at":"2026-05-20T10:21:02Z","pushed_at":"2026-11-15T10:27:26Z","metrics":{"stars":300,"forks":23,"watchers":0,"issues":25,"prs":108,"releases":197,"issues_open":1,"issues_closed":24,"prs_open":1,"prs_closed":13,"prs_merged":94,"discussions":0},"growth":{"day":{"stars":0,"forks":0,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[299,299,299,300,300],"forks":[23,23,23,23,23],"watchers":[0,0,0,0,0],"issues_open":[1,1,1,1,1],"prs_open":[0,0,1,1,1]},"boards":{},"same_owner":[],"issues":[{"number":47,"title":"Help wanted: MLX backend (Part 2) on the safetensors substrate (needs a Mac)","comments":1,"created_at":"2026-06-25T08:06:41Z"}],"discussions":[],"latest_release":{"name":"v0.11.0","tag":"v0.11.0","published_at":"2026-11-03T18:19:01Z","body":"# LlamaStash v0.11.0\n\nThis release adds two CLI verbs, so an agent can manage models without the TUI. `llamastash delete` removes a model's files through the daemon, with the same plan the TUI's `Ctrl+D` shows. `llamastash bench` starts a model once per preset and reports load time, prompt and generation speed, and time to first token. Unified-memory hosts also get a configurable admission reserve, and the HuggingFace pull dialog now opens on the models ranked for your hardware.\n\n## Delete a model from the CLI\n\nAn agent could pull a model but had no way to remove one. Now it can:\n\n```bash\nllamastash delete qwen          # shows the plan, asks [y/N]\nllamastash delete qwen --yes    # deletes without asking\nllamastash delete qwen --yes --json\n```\n\nThe daemon plans the removal and runs it, and the TUI's `Ctrl+D` now goes through the same `delete_model` IPC method. So both clients handle the files a plain `rm -rf` gets wrong: split GGUF shards, an `mmproj` projector another model still pairs with, a separate MTP draft head, a HuggingFace cache repo that still holds a second quant, blobs shared between snapshots, and directory-shaped vLLM rows.\n\nWithout a terminal it refuses unless you pass `--yes`, the same rule `stop --all` follows. The daemon also refuses, removing nothing, for a file a launch still holds and for a plan that has grown past what the client was shown. A refusal or a failed removal exits `75`.\n\n## Per-preset speed numbers with `llamastash bench`\n\n`llamastash bench <model> --preset a,b` starts the model once per preset, waits for it to be ready, sends a fixed prompt `--runs` times (default 3) through the proxy, then stops the launch it started:\n\n```\nPRESET  LOAD  CTX    IN   OUT  PP tok/s  TG tok/s  TTFT   RUNS  NOTE\nsmall   2.2s  4096   969  64   1457      35.2      681ms  2/2   -\nbig     2.2s  16384  969  64   1281      27.1      836ms  2/2   -\n```\n\nThe speeds are the median over the runs that finished, read from the same fields `llamastash requests` prints, so you can compare a knob change without the maintainer scripts in `scripts/bench/`. Without a prompt flag it sends a built-in passage of about 1,000 tokens, prefixed with `Run <n>.` so each run prefills again instead of hitting the KV cache. Every `start` override works here too: with `--ctx 4096`, every row measures a 4096 context.\n\nBench never evicts anything to make room. A preset that does not fit is that row's error, and the other presets are still measured. `--json` gives one result per preset, including the launch's resolved params and the engine's resident memory.\n\n## A configurable memory reserve on unified-memory hosts\n\n`daemon.unified_memory_reserve_percent` sets how much extra memory admission holds back on unified-memory hosts: Apple Silicon, AMD and Intel integrated GPUs, and NVIDIA coherent UMA. The default, `auto`, keeps the existing policy: 25% on Apple Silicon, 0% on other unified-memory hosts. An integer from 0 to 99 overrides it. With 40 GiB available and a 10% reserve, the admission budget is 36 GiB.\n\n```yaml\ndaemon:\n  unified_memory_reserve_percent: auto # or an integer 0..99\n```\n\nIn the TUI, `m` opens an editor with a live preview of the available pool, the reserve and the resulting budget. A save is written to `config.yaml` and applies without a daemon restart. Discrete GPU and CPU-only hosts are unchanged. Thanks to [@howmuchcomputer](https://github.com/howmuchcomputer) for this one ([#126](https://github.com/llamastash/llamastash/pull/126)).\n\n## Other changes worth knowing\n\n- The HuggingFace pull dialog opens on the models the recommender ranks for your hardware, the same list `init` and `llamastash recommend` show, with the recommended quant and a fit mark. An estimated size and fit verdict are marked with `~`.\n- `init --json` and `recommend --json` report each recommendation's numbers as a `facts` object (`params`, `quant`, `peak_bytes`, `host_total`, `bench`, `spill_percent`) instead of a pre-rendered `justification` string. Script","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}