{"full_name":"modelscope/ms-swift","owner":"modelscope","name":"ms-swift","shard":"1","url":"https://github.com/modelscope/ms-swift","description":"Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).","homepage":"https://swift.readthedocs.io/zh-cn/latest/","topics":["llm","lora","sft","multimodal","peft","liger","embedding","grpo","megatron","qwen3","reranker","moe","qwen3-vl","qwen3-omni","deepseek"],"tag_pages":["deepseek","embedding","grpo","llm","lora","moe","multimodal","qwen3"],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/109945100?v=4","created_at":"2023-08-01T15:06:39Z","pushed_at":"2026-11-15T11:59:09Z","metrics":{"stars":16105,"forks":1803,"watchers":55,"issues":5253,"prs":4364,"releases":108,"issues_open":430,"issues_closed":4823,"prs_open":163,"prs_closed":386,"prs_merged":3815,"discussions":0},"growth":{"day":{"stars":1,"forks":0,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[16076,16089,16095,16095,16101,16104,16105],"forks":[1793,1794,1799,1800,1801,1803,1803],"watchers":[null,55,55,55,55,55,55],"issues_open":[null,435,435,435,434,431,430],"prs_open":[null,163,159,160,162,164,163]},"boards":{},"same_owner":[{"full_name":"modelscope/FunASR","name":"FunASR","shard":"1","boards":{}},{"full_name":"modelscope/facechain","name":"facechain","shard":"1","boards":{}},{"full_name":"modelscope/modelscope","name":"modelscope","shard":"1","boards":{}}],"issues":[{"number":4030,"title":"🚀 Best Practices for Training Qwen3/Qwen3-MoE","comments":146,"created_at":"2025-04-28T16:25:01Z"},{"number":7250,"title":"👋Welcome ms-swift v4","comments":62,"created_at":"2025-12-30T03:38:08Z"},{"number":4561,"title":"🍭[Roadmap] ms-swift3.6-3.8","comments":48,"created_at":"2025-06-11T09:38:55Z"},{"number":5507,"title":"qwen3 sft的时候的think","comments":33,"created_at":"2025-08-23T13:59:41Z"},{"number":6476,"title":"qwen3-235b megatron pipeline并行报错","comments":25,"created_at":"2025-11-07T02:09:26Z"},{"number":8053,"title":"grpo训练超时 rollout成功拉起 但是grpo训练报错 XXX:51216 timeour","comments":21,"created_at":"2026-02-15T06:46:33Z"},{"number":6364,"title":"GME模型分别用vLLM部署和swift部署（使用vllm backend ）的服务，返回的Emb结果不一致","comments":21,"created_at":"2025-10-30T10:30:24Z"},{"number":8289,"title":"Best-Practice of Qwen3.5 GRPO training by Megatron-Swift","comments":20,"created_at":"2026-03-11T11:55:37Z"}],"discussions":[],"latest_release":{"name":"v4.6.1","tag":"v4.6.1","published_at":"2026-11-13T09:07:38Z","body":"# v4.6.1\r\n\r\n## New Models\r\n\r\n- [NVIDIA-Nemotron-3.5-Super-VL-120B-A12B](https://modelscope.cn/models/nv-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16)(#10377)\r\n- [NVIDIA-Nemotron-3.5-Transcribe-30B-A4B](https://modelscope.cn/models/nv-community/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B)(#10458)\r\n- [Unlimited-OCR-2](https://modelscope.cn/models/PaddlePaddle/Unlimited-OCR-2)(#10448)\r\n- [MiniCPM-OCR-1.0](https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0)(#10414)\r\n- [Olmo-3-7B-Think](https://modelscope.cn/models/allenai/Olmo-3-7B-Think)(#10371)\r\n- [flint](https://modelscope.cn/models/RecursiveSI/flint)(#10482)\r\n- [EmbeddingGemma-2](https://modelscope.cn/models/google/embeddinggemma-2)(#10392)\r\n- Decision / System-1 family: [THX-02](https://modelscope.cn/models/HAL-X/THX-02) (#10398), [kev-0.8b/4b/9b](https://huggingface.co/jaredpalmer) (#10407), [laya](https://modelscope.cn/models/convaiinnovations/laya) (#10445) and [StartLux-Decision-v1.1 4B/9B/27B-AutoThink](https://modelscope.cn/models/StartLuxAI/StartLux-Decision-v1.1-27B-AutoThink) (#10479).\r\n\r\n## Decision Task (typed-decision / JEV family)\r\n\r\n- Four new decision heads with templates, alignment tests and `examples/train/decision/*.sh` recipes: THX-02 (#10398), `jaredpalmer/kev-*` pointer heads (#10407), Laya (#10445) and StartLux (#10479).\r\n- Throughput: whole-batch forward for Clef/Kev scoring heads (#10551) and multi-question forward for Kev (#10494).\r\n- Correctness & calibration: two-stage JEV temperature calibration callback (#10399), gradients preserved for confidently-wrong predictions (#10355), multimodal base modes with text-only data (#10420), injected output embeddings kept out of the head module tree (#10540).\r\n- The THX-02-specific packing check is replaced by the generic `support_padding_free` flag on `ModelMeta` (#10480).\r\n- `docs/*/BestPractices/Decision.md` rewritten as the entry point for the family.\r\n\r\n## New Features & Compatibility\r\n\r\n- **Entropack flexible quantization** for QLoRA-style training, with a new best-practice doc (`Flexble-Quantization-With-Entropack.md`) and `examples/train/qlora/entropack.sh`. (#10415)\r\n- **Megatron parallel-argument preflight**: TP/PP/CP/EP/ETP divisibility is checked at the Swift argument entry point before process-group initialization, with actionable error messages; the Transformers backend and the Megatron export path are unchanged. (#10393)\r\n- **Gemma 4**: packing / `padding_free` for text-only data on the Megatron backend. (#10542)\r\n- **Huawei Ascend**: Megatron training migrated to Megatron-Core 0.18 with `MegatronAdaptor` + TENPU (#10087); standalone environment check script `examples/ascend/check_env.py` (#10390).\r\n\r\n## Megatron-SWIFT & MTP\r\n\r\n- `--mtp_num_layers` now defaults to `auto` and follows the model checkpoint; `--mtp_shared_weights` defaults to automatic detection (enabled when the checkpoint holds fewer MTP layers than the configured depth). To add MTP to a base model that ships none, set `--mtp_num_layers N --mtp_allow_random_init true`. (#10416, requires mcore-bridge >= 1.7.0)\r\n- Async-checkpoint import fixed against the latest Megatron dev branch. (#10421)\r\n\r\n## Performance\r\n\r\n- `selective_log_softmax` chunked instead of looping per token. (#10337)\r\n- Reduced custom and chunked cross-entropy work in SFT. (#10353)\r\n- Lower NPU gradient, RMSNorm and MoE-permutation overhead in Megatron training. (#10513)\r\n\r\n## Security\r\n\r\n- `external_plugins` declared in a checkpoint's `args.json` is no longer restored silently on resume. (#10422)\r\n- HMAC authentication for ZMQ control messages in the checkpoint engine, abstracted to cover both HCCL and NCCL transports. (#10373, #10381)\r\n- `load_file` now applies `SafeMediaPath` before opening a local media path, so `SWIFT_MEDIA_ALLOWED_DIRS` can no longer be bypassed through template image/audio loading. (#10370)\r\n\r\n## Bug Fixes\r\n\r\n- **RL**: SP state preserved during async server rollout (#10426); GRPO logging by gathering/caching rollout records (#10427); `ov","is_prerelease":false},"readme":true,"readme_zh":true,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}