{"full_name":"lyogavin/airllm","owner":"lyogavin","name":"airllm","shard":"1","url":"https://github.com/lyogavin/airllm","description":"AirLLM 2.8T LLM inference with single 4GB GPU","homepage":"","topics":["chinese-nlp","finetune","generative-ai","instruct-gpt","instruction-set","llama","llm","lora","open-models","open-source","open-source-models","qlora","chinese-llm"],"tag_pages":["generative-ai","llama","llm","lora","open-source"],"language":"Jupyter Notebook","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/1113905?u=fd3d25f2d8e597e495d8ffd10d438ad22eb0e94e&v=4","created_at":"2023-06-12T16:28:41Z","pushed_at":"2026-11-15T06:13:36Z","metrics":{"stars":42457,"forks":4386,"watchers":333,"issues":265,"prs":89,"releases":6,"issues_open":111,"issues_closed":154,"prs_open":46,"prs_closed":15,"prs_merged":28,"discussions":28},"growth":{"day":{"stars":78,"forks":17,"base_day":"2026-11-15"},"week":{"stars":424,"forks":58,"base_day":"2026-11-09"}},"history":{"dates":["2026-10-27","2026-10-28","2026-10-29","2026-10-30","2026-10-31","2026-11-01","2026-11-02","2026-11-03","2026-11-04","2026-11-09","2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[40536,40556,40756,40978,41105,41227,41333,41603,41666,42033,42081,42172,42237,42256,42318,42379,42457],"forks":[4136,4138,4157,4179,4199,4211,4226,4270,4279,4328,4335,4343,4352,4358,4365,4369,4386],"watchers":[319,314,317,318,321,321,322,323,324,326,326,328,331,333,333,333,333],"issues_open":[null,null,null,null,null,null,null,null,null,null,null,111,111,111,111,111,111],"prs_open":[null,null,null,null,null,null,null,null,null,null,null,45,45,45,46,46,46]},"boards":{},"same_owner":[],"issues":[{"number":258,"title":"[Partially solved] What's the problem with these AI toys? Not one that works at fist try","comments":19,"created_at":"2026-02-11T20:56:10Z"},{"number":318,"title":"AMD ROCM support?","comments":1,"created_at":"2026-08-01T12:09:34Z"},{"number":301,"title":"run this with windows amd 9070 XT ROCm 7.12  VERRY SLOW","comments":3,"created_at":"2026-07-06T12:39:10Z"},{"number":330,"title":"3x speedup compression claim is false/outdated","comments":2,"created_at":"2026-08-04T22:18:47Z"},{"number":319,"title":"grok claude","comments":0,"created_at":"2026-08-01T21:54:28Z"},{"number":44,"title":"More clever batching of layers","comments":13,"created_at":"2023-11-30T11:52:03Z"},{"number":192,"title":"it is run","comments":11,"created_at":"2024-10-24T20:25:50Z"},{"number":328,"title":"Login button is not working on Homepage","comments":0,"created_at":"2026-08-04T10:17:52Z"}],"discussions":[{"number":400,"title":"Intelligent Memory Management for AirLLM","comments":0,"category":"Ideas","created_at":"2026-11-14T18:09:41Z"},{"number":391,"title":"Show and tell: an interactive visualization of the layer-streaming memory/time tradeoff","comments":0,"category":"Show and tell","created_at":"2026-10-27T08:28:18Z"},{"number":390,"title":"artefact english coach","comments":0,"category":"General","created_at":"2026-10-26T14:38:25Z"},{"number":376,"title":"Collab proposal: optional diagnostics + repro bench (no new deps)","comments":1,"category":"Ideas","created_at":"2026-10-11T11:17:33Z"},{"number":361,"title":"An idea regarding the diffusion models","comments":1,"category":"General","created_at":"2026-09-06T13:24:00Z"},{"number":338,"title":"AirWrapperLLM for AirLLM","comments":0,"category":"Show and tell","created_at":"2026-08-08T15:43:25Z"},{"number":320,"title":"Wouldn’t this be very slow ?","comments":3,"category":"Q&A","created_at":"2026-08-02T08:35:10Z"},{"number":314,"title":"[Bug] Qwen3-235B-A22B (BF16 & FP8) inference fails — streaming loader cannot handle fused MoE expert tensors (Qwen3MoeExperts / FP8Experts)","comments":0,"category":"General","created_at":"2026-07-27T14:16:47Z"}],"latest_release":{"name":"AirLLM v4.0.0 — train Qwen3.8-Flash-Next under 6GB","tag":"v4.0.0","published_at":"2026-09-05T15:43:11Z","body":"Streamed LoRA: frozen weights stay on disk, adapters stay on GPU. **Qwen3.8-Flash-Next** trains under **6GB** (RTX 3060 Ti); **Qwen3.8-27B** trains in **~2GB** at seq 512.\n\n```bash\npip install -U airllm\npip install git+https://github.com/huggingface/transformers.git\n```\n\nPoint `--data` at a JSONL file (`{\"text\": \"...\"}` or `{\"prompt\": \"...\", \"completion\": \"...\"}`):\n\n```bash\npython air_llm/examples/train_qwen38_flash_next_lora.py   --data my_data.jsonl   --seq-len 512   --epochs 1   --save-adapter qwen38-flash-next-lora.pt\n```\n\nFlash-Next still needs in-tree `qwen4_exp` (GitHub `transformers` main today).","is_prerelease":false},"readme":true,"readme_zh":false,"news":[{"title":"AirLLM 70B inference with single 4GB GPU","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=49154228","source":"Hacker News","lang":"en","date":"2026-08-03","points":232,"comments":84},{"title":"AirLLM: Inference 2.8T Kimi K3 on a single 4GB GPU","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=49142538","source":"Hacker News","lang":"en","date":"2026-08-02","points":2,"comments":0},{"title":"AirLLM optimizes inference memory usage","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=47293735","source":"Hacker News","lang":"en","date":"2026-03-08","points":2,"comments":0},{"title":"AirLLMで4GBのGPUに70Bモデルは動くのか、1トークンにかかる時間を試算した","url":"https://note.com/snake_dragon/n/nb4f1b7ebefeb","source":"note","lang":"zh","date":"2026-10-28"},{"title":"为什么你的 AirLLM 比别人慢三倍？一个 4GB 老显卡玩家的调优踩坑全复盘","url":"https://blog.csdn.net/gitblog_01114/article/details/168541349","source":"blog.csdn","lang":"zh","date":"2026-10-20"},{"title":"刚刚冲上 GitHub 日榜第7：AirLLM 一天狂揽 541 星，三年老项目为何突然翻红？","url":"https://blog.csdn.net/gitblog_00634/article/details/168425887","source":"blog.csdn","lang":"zh","date":"2026-10-19"},{"title":"这周 GitHub 上涨星最猛的 10 个 AI 开源项目","url":"https://www.bilibili.com/video/BV1UAn36aEX3/","source":"bilibili","lang":"zh","date":"2026-10-17"},{"title":"AirLLM: Große KI-Modelle auf schwachen Grafikkarten starten","url":"https://www.heise.de/ratgeber/AirLLM-Grosse-KI-Modelle-auf-schwachen-Grafikkarten-starten-11476015.html?seite=all","source":"heise online","lang":"en","date":"2026-10-06"},{"title":"I want to use a heavy LLM locally, but I only have 12GB of VRAM! For you, there's AirLLM? I investigated whether it's actually usable.","url":"https://note.com/humble_bobcat51/n/na7f9589db317?hl=en","source":"note","lang":"en","date":"2026-02-09"},{"title":"本周最猛的10个 AI 开源项目","url":"https://news.google.com/rss/articles/CBMiVkFVX3lxTE9WSkYwZnZsLWthRVVfV1B4bno1WWtwQWFFR3JQakxmYkhPUHYxNWIyQWpCZF8wMkpnVWJPb3U3dmx4Zk1ZcVRQV2FzbmVXU3dIY1VXTWlB?oc=5","source":"腾讯新闻","lang":"zh","date":"2026-10-18"}],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}