{"full_name":"FareedKhan-dev/kimi-k3-in-c","owner":"FareedKhan-dev","name":"kimi-k3-in-c","shard":"1","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","description":"A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.","homepage":"https://medium.com/@fareedkhandev/building-kimi-k3-in-c-to-run-a-2-8t-model-on-consumer-hardware-a5792cbf3b59","topics":["avx2","c99","cpu-inference","deep-learning","from-scratch","inference-engine","kimi-k3","linear-attention","llm","llm-inference","machine-learning","memory-efficient","mixture-of-experts","moe","mxfp4"],"tag_pages":["avx2","deep-learning","from-scratch","llm","llm-inference","machine-learning","mixture-of-experts","moe"],"language":"C","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/63067900?u=09fcee42c39e5a66924e5012f966d75b979b7041&v=4","created_at":"2026-08-01T09:29:38Z","pushed_at":"2026-11-15T17:03:29Z","metrics":{"stars":12133,"forks":1876,"watchers":105,"issues":19,"prs":96,"releases":4,"issues_open":2,"issues_closed":17,"prs_open":0,"prs_closed":74,"prs_merged":22,"discussions":0},"growth":{"day":{"stars":65,"forks":16,"base_day":"2026-11-15"},"week":{"stars":769,"forks":99,"base_day":"2026-11-09"},"month":{"stars":3123,"forks":400,"base_day":"2026-10-17"}},"history":{"dates":["2026-09-06","2026-09-07","2026-09-08","2026-09-09","2026-09-10","2026-09-11","2026-09-12","2026-09-13","2026-09-14","2026-09-15","2026-09-16","2026-09-17","2026-09-18","2026-09-19","2026-09-20","2026-09-21","2026-09-22","2026-09-23","2026-09-24","2026-09-25","2026-09-26","2026-09-27","2026-09-28","2026-09-29","2026-09-30","2026-10-01","2026-10-02","2026-10-03","2026-10-04","2026-10-05","2026-10-06","2026-10-07","2026-10-08","2026-10-09","2026-10-10","2026-10-11","2026-10-12","2026-10-13","2026-10-14","2026-10-15","2026-10-16","2026-10-17","2026-10-18","2026-10-19","2026-10-20","2026-10-21","2026-10-22","2026-10-23","2026-10-24","2026-10-25","2026-10-26","2026-10-27","2026-10-28","2026-10-29","2026-10-30","2026-10-31","2026-11-01","2026-11-02","2026-11-03","2026-11-04","2026-11-09","2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[6396,6470,6514,6587,6628,6669,6715,6953,7023,7063,7112,7137,7182,7214,7286,7505,7567,7761,7911,7954,7993,8012,8041,8082,8133,8267,8432,8530,8645,8677,8761,8795,8829,8836,8864,8880,8909,8926,8940,8955,8969,9010,9028,9045,9057,9073,9086,9108,9127,9143,9157,9168,9178,9190,9199,9216,9231,9251,9330,9783,11364,11528,11773,11877,11911,11973,12068,12133],"forks":[1041,1053,1061,1074,1079,1086,1095,1126,1132,1143,1154,1160,1172,1181,1185,1205,1215,1243,1276,1280,1282,1288,1297,1301,1313,1328,1347,1371,1394,1399,1418,1424,1435,1438,1442,1443,1456,1459,1465,1468,1470,1476,1478,1479,1481,1481,1483,1486,1487,1491,1493,1495,1500,1502,1502,1506,1508,1509,1522,1573,1777,1796,1819,1832,1838,1853,1860,1876],"watchers":[62,62,60,61,62,62,64,66,66,66,66,65,67,67,67,67,66,68,69,69,70,71,71,71,71,73,72,72,73,74,75,76,76,75,75,75,76,76,76,76,77,79,80,81,81,81,81,81,81,81,81,82,81,81,81,81,81,81,81,86,96,96,102,103,103,103,104,105],"issues_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,2,2,2,2,2,2],"prs_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,31,31,31,31,31,0]},"boards":{"monthly":182,"weekly_rate":172,"monthly_rate":101},"same_owner":[{"full_name":"FareedKhan-dev/train-llm-from-scratch","name":"train-llm-from-scratch","shard":"1","boards":{}}],"issues":[{"number":3,"title":"Support for Quantized / Storage-Efficient Kimi K3 Checkpoints","comments":7,"created_at":"2026-08-03T04:50:16Z"},{"number":23,"title":"I have a 5070TI with 16 GB Mem","comments":3,"created_at":"2026-08-10T10:11:35Z"}],"discussions":[],"latest_release":{"name":"v2.0.0","tag":"v2.0.0","published_at":"2026-11-15T16:50:47Z","body":"Long prompts stop being a one-core job, and a large round of fixes, most of them from outside contributors, makes everything that loads, parses, measures or certifies refuse a wrong input instead of quietly using it. Output is unchanged wherever it was already defined: every oracle gate still matches the PyTorch reference exactly, at any thread count.\n\n**Compatibility notes, which are why this is 2.0.0:**\n\n- `--save-state` files are now version 3, with the header inside the checksum. A version 2 file is refused rather than read; regenerate it with `--gen 0 --incremental --save-state`.\n- MLA scratch grew: the score term of `k3_mla_scratch_cached()` is now H*T floats. Code that sizes scratch through the helpers is unaffected. `K3Cache` gained fields, so rebuild anything that embeds it.\n- Chat sampling with both `--top-k` and `--top-p` measures the nucleus after top-k, as the Hugging Face and vLLM samplers do, so a fixed-seed run using both flags samples differently.\n- Malformed numeric flags (`--gen 1junk`, an empty `--seed`, `--stop-id 4294967296`) exit 2 instead of being parsed leniently, and a prompt longer than 32,768 ids is refused instead of truncated.\n\n## Faster, output unchanged\n\n- MLA attention runs across heads in parallel. It was the one serial O(T^2) loop left in the engine. Decode at 4,096 positions at the released head geometry: 303.3 ms per MLA layer on one thread, 38.6 ms on 16, so 7.28 s/token becomes 0.93 s/token over the 24 MLA layers, with identical output hashes (Intel Core Ultra 7 255H, `make bench`). At 32k positions the old loop alone cost about 58 s per token on that machine.\n\n## New\n\n- Memory admission respects cgroup limits, so a run inside a container or a `systemd-run -p MemoryMax=` scope is refused with both numbers instead of being OOM-killed.\n- The tokenizer loader has a weightless gate on a synthetic vocabulary, so it runs in CI for the first time.\n- [docs/TERMUX.md](https://github.com/FareedKhan-dev/kimi-k3-in-c/blob/v2.0.0/docs/TERMUX.md): building and running the weightless suite on Android ARM64.\n\n## Fixed\n\n- Loaders refuse what they used to accept: a corrupt or truncated `tiktoken.model`, an expert of the wrong shape, an E8M0 scale carrying the NaN code, a split expert read whose middle chunk came back short, and a state header that disagrees with the config.\n- `--layers 1`, the documented ~7 GB development path, now runs.\n- A failed `--out`, logits dump or trace write exits 3 instead of 0.\n- The chat sampler refuses NaN and infinite logits instead of turning them into a token.\n- The tools that certify results can no longer certify nothing: conformance, logit comparison, the benchmark harnesses, device bandwidth, quantization, packing, budgeting and cache replay all refuse empty, stale or malformed evidence. The CLI only records the expert request trace when asked, and `make asan` no longer relinks objects that were never instrumented.\n- Windows: `make test` failed in `test_chat` whenever the released tokenizer was present; `make tok TOK_FILES=...` ignored `TOK_FILES`. The macOS CI job runs again, on `macos-15`.\n\n## Verification\n\nOn the released checkpoint, on a 28-core AMD EPYC 9554:\n\n- The full 93-layer golden run (`tests/fixtures/golden`): logits byte-identical on 1.1.0 and 2.0.0, and byte-identical to the logits the original campaign recorded. Token 2494, and agreement with the PyTorch reference at 5.2e-7 relative error (budget 5.1e-4), top-10 overlap 10/10.\n- The first eight layers, both MLA layers among them, with a 613-token prompt: 1.1.0 and 2.0.0 produce byte-identical logits and the same tokens, with no expert drops.\n- Real layer 1 (KDA) and layer 3 (MLA) against the PyTorch reference: every stage at float32 rounding, about 1e-7 relative error against a 5e-4 budget.\n\n`make test`, ctest, both sanitizer jobs and the strict-warnings build are green on Linux, macOS and Windows, and every oracle gate matches the PyTorch reference exactly. The weightless suite also ran with the released ","is_prerelease":false},"readme":true,"readme_zh":false,"news":[{"title":"Kimi K3 2.78T on One CPU with 8GB RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=49181439","source":"Hacker News","lang":"en","date":"2026-08-05","points":4,"comments":0},{"title":"Kimi-k3-in-C: A 2.78T-parameter model. No GPU. 8 GB of RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=50357277","source":"Hacker News","lang":"en","date":"2026-11-03","points":2,"comments":1},{"title":"Kimi-k3-in-C, 2.78T-parameter model, one CPU, 8GB RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=49167975","source":"Hacker News","lang":"en","date":"2026-08-04","points":2,"comments":0},{"title":"kimi-k3-in-c","url":"https://www.53ai.com/AIWiki/pQX3kl.html","source":"53ai","lang":"en","date":"2026-10-09"},{"title":"kimi-k3-in-c 量化实测：为什么拒绝量化 Trunk？int8 损失1%、int4 损失17%的完整数据解析","url":"https://blog.csdn.net/gitblog_00359/article/details/155462282","source":"blog.csdn","lang":"zh","date":"2026-09-01"},{"title":"Kimi K3 开放权重全景解读:首个开源 3T 级多模态模型的架构与本地部署","url":"https://blog.csdn.net/Kartist139/article/details/163684634","source":"blog.csdn","lang":"zh","date":"2026-08-12"},{"title":"神了，8G 内存能本地跑 Kimi K3 了。。。","url":"https://news.google.com/rss/articles/CBMiU0FVX3lxTE8yQUtzajlFOXFUd1BhMGVidF90S3h1SGpRbDJXVDQ3Zmp6Ty02eHFVSmtWLU8weHFFNnU3RGpvNEFvRU4wVGlZTk1UOVg3VUpUNWpr?oc=5","source":"电子工程专辑EE Times China","lang":"zh","date":"2026-09-04"},{"title":"8GB 内存也能跑 Kimi K3？2026 本地部署大模型配置全指南","url":"https://news.google.com/rss/articles/CBMiQ0FVX3lxTE8tSzNWcUl5NXN0MHZtNmJ5V213UkdwX3JXS2g1dnJXYUI2QXZBRmljdVU4dS03eUg5ajZ0Y05ERXhGR1E?oc=5","source":"爱范儿","lang":"zh","date":"2026-08-05"},{"title":"Kimi K3: 2.78 Trillion Parameters on One CPU, 8 GB RAM","url":"https://news.google.com/rss/articles/CBMihAFBVV95cUxQc3ZndzlvNE9XRE9ENmp3NUZWalppR0RuSC1naTFTYW04bHV0YVROVE9hWHFJRi1KR3RoT3F0YzJ5eXkzR016a3h4UERheUxubHpoY0Ixd2dST3FfbWdmOEpOdHdBSmtMT3pjWmZLcEUyVHZTTmZ5M2d1N1Nxck5acHVWNHI?oc=5","source":"Pasquale Pillitteri","lang":"en","date":"2026-08-03"},{"title":"Kimi K3：单颗 CPU、8 GB 内存跑通 2.78 万亿参数","url":"https://news.google.com/rss/articles/CBMihgFBVV95cUxONjYtVE5paDJ6NzZ2Z3Y0eXR6UTh6Z05JVlRMdldTaVNzc1ZBUFRMbUFrLVJLR0xaOUhfZ1dTbXJ3TDhnV3BqbHNxN3VhXzBQYzZkYU8wbVJILWF5X2F6aUNRc2MyMVo1eG1idWprdGlqdmZQZXhJX2QtdVJPUmZlaDdfVnlLdw?oc=5","source":"Pasquale Pillitteri","lang":"zh","date":"2026-08-03"}],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}