{"full_name":"kvcache-ai/ktransformers","owner":"kvcache-ai","name":"ktransformers","shard":"1","url":"https://github.com/kvcache-ai/ktransformers","description":"A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations","homepage":"https://kvcache-ai.github.io/ktransformers/","topics":[],"tag_pages":[],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/170996193?v=4","created_at":"2024-07-26T07:18:28Z","pushed_at":"2026-11-13T07:51:35Z","metrics":{"stars":19675,"forks":1602,"watchers":120,"issues":1317,"prs":813,"releases":34,"issues_open":472,"issues_closed":845,"prs_open":67,"prs_closed":97,"prs_merged":649,"discussions":0},"growth":{"day":{"stars":4,"forks":0,"base_day":"2026-11-15"},"week":{"stars":17,"forks":0,"base_day":"2026-11-04"},"month":{"stars":63,"forks":0,"base_day":"2026-10-17"}},"history":{"dates":["2026-09-06","2026-09-07","2026-09-08","2026-09-09","2026-09-10","2026-09-11","2026-09-12","2026-09-13","2026-09-14","2026-09-15","2026-09-16","2026-09-17","2026-09-18","2026-09-19","2026-09-20","2026-09-21","2026-09-22","2026-09-23","2026-09-24","2026-09-25","2026-09-26","2026-09-27","2026-09-28","2026-09-29","2026-09-30","2026-10-01","2026-10-02","2026-10-03","2026-10-04","2026-10-05","2026-10-06","2026-10-07","2026-10-08","2026-10-09","2026-10-10","2026-10-11","2026-10-12","2026-10-13","2026-10-14","2026-10-15","2026-10-16","2026-10-17","2026-10-18","2026-10-19","2026-10-20","2026-10-21","2026-10-22","2026-10-23","2026-10-24","2026-10-25","2026-10-26","2026-10-27","2026-10-28","2026-10-29","2026-10-30","2026-10-31","2026-11-01","2026-11-02","2026-11-03","2026-11-04","2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[19467,19470,19478,19481,19484,19489,19494,19503,19508,19514,19518,19520,19521,19524,19521,19527,19532,19532,19535,19536,19540,19538,19541,19545,19544,19549,19554,19557,19559,19561,19566,19569,19569,19571,19576,19589,19592,19600,19601,19607,19603,19612,19613,19614,19619,19616,19620,19622,19625,19630,19632,19637,19641,19649,19652,19649,19647,19646,19655,19658,19659,19661,19661,19662,19667,19671,19675],"forks":[1564,1562,1565,1567,1571,1571,1573,1573,1571,1571,1572,1572,1573,1575,1576,1577,1576,1576,1576,1575,1578,1578,1580,1581,1581,1582,1585,1586,1585,1585,1590,1590,1591,1592,1592,1597,1600,1602,1602,1603,1604,1602,1602,1603,1604,1601,1602,1602,1602,1601,1600,1598,1597,1597,1597,1597,1598,1600,1601,1602,1601,1599,1600,1600,1601,1602,1602],"watchers":[125,126,126,125,125,125,125,125,125,125,125,125,125,125,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,124,125,125,125,125,125,125,125,125,125,125,125,120,120,121,121,121,121,121,121,121,121,121,null,120,120,120,120,120,120],"issues_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,473,473,473,472,472,472],"prs_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,68,67,68,67,67,67]},"boards":{},"same_owner":[],"issues":[{"number":1417,"title":"[Bug] deepseek v3/r1 with full context with balance_serve backend","comments":74,"created_at":"2025-07-03T12:34:10Z"},{"number":150,"title":"再次提一下，能否支持2080ti","comments":44,"created_at":"2025-02-12T04:12:57Z"},{"number":1785,"title":"SGLang does not support kt-kernel","comments":37,"created_at":"2026-01-06T12:17:23Z"},{"number":514,"title":"这个模型跑起来，如何使用呀，如何和其他系统进行对接，比如dify 国产的maxkb上如何使用了","comments":31,"created_at":"2025-02-19T15:43:50Z"},{"number":482,"title":"复现了，但DeepSeek-R1-Q4_K_M跑起来速度非常慢，只有约1.5token/s，请问是我配置的原因么？","comments":30,"created_at":"2025-02-19T02:47:07Z"},{"number":1598,"title":"[Feature] Support Native K2 Thinking","comments":28,"created_at":"2025-11-12T03:00:03Z"},{"number":1490,"title":"is this project dead?","comments":28,"created_at":"2025-08-31T20:29:26Z"},{"number":1730,"title":"Performance Query: Is ~9.5 tokens/s normal for Kimi-K2-Thinking-Native on Xeon 6960P (MRDIMM 8800) + RTX 6000 Blackwell?","comments":23,"created_at":"2025-12-18T05:01:45Z"}],"discussions":[],"latest_release":{"name":"KTransformers v0.7.2","tag":"v0.7.2","published_at":"2026-10-15T07:09:13Z","body":"# KTransformers v0.7.2: DeepSeek-V4-Flash LoRA Fine-Tuning\r\n\r\nKTransformers v0.7.2 adds LoRA fine-tuning for the April DeepSeek-V4-Flash checkpoint. Routed experts remain in native MXFP4 on CPU, while attention and other non-expert modules run on GPU.\r\n\r\n## 🚀 Highlights\r\n\r\n- Native MXFP4 routed-expert LoRA forward and backward, without expanding the frozen expert weights to BF16.\r\n- LLaMA-Factory SFT integration with checkpoint saving and SGLang adapter inference.\r\n- Python 3.11 and 3.12 Linux x86-64 wheels.\r\n\r\n## 📦 Installation\r\n\r\nFor training:\r\n\r\n```bash\r\npython -m pip install 'ktransformers[sft]==0.7.2'\r\n```\r\n\r\nFor inference:\r\n\r\n```bash\r\npython -m pip install 'ktransformers[sglang]==0.7.2'\r\n```\r\n\r\n## 🛠️ Main-Branch Updates\r\n- [KTransformers-FineTune preprint v1.1](https://github.com/kvcache-ai/ktransformers/commit/6160538c81fc1d2cbc1952302a6d4b77a8adcd2e) upload and update our paper to the latest.\r\n- [CPU BF16 gradient-norm reduction](https://github.com/kvcache-ai/ktransformers/commit/c76ff29b2aae025c6a9cb139f83426a1156c618c) adds a native sum-of-squares primitive for KT-owned gradients without materializing an FP32 tensor, which reduces the grad clip overhead of full fine-tuning.\r\n\r\n\r\n## 🌟 Contributors\r\n\r\nThanks to all human code and documentation contributors since v0.7.1: @JimmyPeilinLi, @yyj6666667, @05yuki, @losewayy, @Dashener2, @pratikgx, @MohammadHijjawi97, and @Anai-Guo. Thanks also to everyone who reviewed and tested these changes.\r\n","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}