{"full_name":"sgl-project/sglang","owner":"sgl-project","name":"sglang","shard":"1","url":"https://github.com/sgl-project/sglang","description":"SGLang is a high-performance serving framework for large language models and multimodal models.","homepage":"https://sglang.io","topics":["cuda","inference","llama","llm","moe","transformer","vlm","deepseek","blackwell","gpt-oss","diffusion","attention","glm","minimax","qwen"],"tag_pages":["attention","blackwell","cuda","deepseek","diffusion","glm","gpt-oss","inference","llama","llm","minimax","moe","qwen","transformer","vlm"],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/147780389?v=4","created_at":"2024-01-08T04:15:52Z","pushed_at":"2026-11-15T18:27:46Z","metrics":{"stars":38901,"forks":10510,"watchers":185,"issues":8429,"prs":41621,"releases":64,"issues_open":976,"issues_closed":7453,"prs_open":5813,"prs_closed":12581,"prs_merged":23227,"discussions":425},"growth":{"day":{"stars":31,"forks":28,"base_day":"2026-11-15"},"week":{"stars":296,"forks":221,"base_day":"2026-11-09"}},"history":{"dates":["2026-11-09","2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[38605,38643,38744,38789,38796,38830,38870,38901],"forks":[10289,10315,10371,10407,10416,10435,10482,10510],"watchers":[183,183,183,184,184,184,184,185],"issues_open":[null,null,947,959,968,964,983,976],"prs_open":[null,null,5715,5750,5778,5843,5848,5813]},"boards":{"daily_forks":181},"same_owner":[{"full_name":"sgl-project/mini-sglang","name":"mini-sglang","shard":"1","boards":{}},{"full_name":"sgl-project/sglang-omni","name":"sglang-omni","shard":"1","boards":{}}],"issues":[{"number":26340,"title":"CUDA Coredump Tracker","comments":367,"created_at":"2026-05-26T03:03:57Z"},{"number":42752,"title":"[CI] Flaky tests and CI infrastructure failures seen while babysitting PRs","comments":208,"created_at":"2026-10-06T10:42:20Z"},{"number":21302,"title":"[AITER-Upgrade] PR readiness","comments":45,"created_at":"2026-03-24T09:28:38Z"},{"number":33522,"title":"[Roadmap]Fast Engine Recovery: Weight Cache Daemon","comments":35,"created_at":"2026-08-04T09:05:53Z"},{"number":39499,"title":"[Roadmap] SGLang dLLM Serving","comments":31,"created_at":"2026-09-15T01:21:47Z"},{"number":19137,"title":"[Roadmap] Apple Device Support (2026 Q2)","comments":30,"created_at":"2026-02-22T03:21:15Z"},{"number":42012,"title":"[Bug] GLM-5.3-Flash on SM120: fa4 attention backend crashes at CUDA-graph capture (hybrid extend reshape) — triton is the only working backend","comments":28,"created_at":"2026-10-01T06:22:43Z"},{"number":33422,"title":"[Perf] DeepSeek-V4 dsv4 prefill on SM120 (4x RTX PRO 6000, TP=4): ~2-7K tok/s vs ~12.5K reported via vLLM/marlin on identical hardware — expected? tuning guidance?","comments":27,"created_at":"2026-08-03T19:29:12Z"}],"discussions":[{"number":49996,"title":"loraspectra & mcp-slim: Zero-dependency CLI tools for SGLang LoRA SVD rank compression/merging and MCP schema optimization","comments":0,"category":"Show and tell","created_at":"2026-11-12T02:35:36Z"},{"number":49772,"title":"FastH3-Omni是否能够支持SM12.0（GPU5090）开启VSA？","comments":1,"category":"Q&A","created_at":"2026-11-11T07:48:52Z"},{"number":47933,"title":"Should sglang load lazily like vllm?","comments":0,"category":"General","created_at":"2026-11-02T14:09:12Z"},{"number":47143,"title":"VRAM & KV-Cache Sizing Calculator for High-Throughput Serving & RadixAttention Planning (Zero-Dependency CLI & WebApp)","comments":0,"category":"Show and tell","created_at":"2026-10-28T21:10:00Z"},{"number":46804,"title":"Cookbook: are consumer SM89 (RTX 40-series, 16 GB) notes welcome on the Qwen3 page?","comments":3,"category":"Q&A","created_at":"2026-10-27T09:13:49Z"},{"number":46591,"title":"[Case Study] Agent Context Compaction Breaks RadixAttention Prefix Caching (10.1% -> 99.6% Hit Rate)","comments":0,"category":"Show and tell","created_at":"2026-10-26T12:13:53Z"},{"number":46590,"title":"[Case Study] Agent Context Compaction Breaks RadixAttention Prefix Caching (10.1% -> 99.6% Hit Rate)","comments":0,"category":"Show and tell","created_at":"2026-10-26T12:09:05Z"},{"number":45932,"title":"DeepSeek-V4.1-Flash-1017 on two 16 GB RTX 5070 Ti with the routed experts on the CPU (SGLang + KTransformers, 1M context)","comments":0,"category":"Show and tell","created_at":"2026-10-23T01:32:49Z"}],"latest_release":{"name":"v0.5.23","tag":"v0.5.23","published_at":"2026-11-02T22:14:10Z","body":"# Highlights\r\n\r\n*1007 PRs from 257 contributors.*\r\n\r\n## New models\r\n\r\n| Model | Type | Cookbook |\r\n|---|---|---|\r\n| Muse Spark Open | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen) |\r\n| Beam-1.0 | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Reflection/Beam-1_0) |\r\n| MiniMax-M3.1-Flash | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1) |\r\n| MiniCPM-V 4.7 | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/OpenBMB/MiniCPM-V-4_7) |\r\n| Kev | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Kev/kev) |\r\n| Cosmos3-Nano-Sim (Bimanual, Transfer, Auto) | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3) |\r\n| FastH3 Omni | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3) |\r\n| FLUX.3 Image Klein | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/FLUX/FLUX.3) |\r\n| HY-Nomas | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/HY-Nomas/HY-Nomas) |\r\n| MOSS-Image | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/OpenMOSS/MOSS-Image) |\r\n\r\n## What's new\r\n\r\n* DeepSeek-V4.1 on GB300 gets up to 1.5x more prefill throughput under load and about 28% lower first-token latency on 32K to 64K prompts ([#44403](https://github.com/sgl-project/sglang/pull/44403), [#45059](https://github.com/sgl-project/sglang/pull/45059), [#45107](https://github.com/sgl-project/sglang/pull/45107)).\r\n* SGLang low-latency mode: the opt-in NVIDIA [megakernel](https://github.com/sgl-project/sgl-nvidia-mega) and AMD [monokernel](https://github.com/sgl-project/sgl-amd-mono) plugins run DeepSeek-V4.1-Flash decode as fused persistent kernels, for up to 44% faster single-user decode on GB300 and 75% on MI350X ([#44945](https://github.com/sgl-project/sglang/pull/44945), [#45450](https://github.com/sgl-project/sglang/pull/45450), [#44353](https://github.com/sgl-project/sglang/pull/44353), [#44621](https://github.com/sgl-project/sglang/pull/44621)).\r\n* MoE models such as DeepSeek-V4.1-Flash, GLM-5.3-Flash and gpt-oss reach the first token 30% to 58% sooner on 8K prompts on H200 ([#43363](https://github.com/sgl-project/sglang/pull/43363)).\r\n* Paged Experts (opt-in) runs MoE models larger than GPU memory on a single consumer GPU, up to 30x faster than CPU offload ([#42557](https://github.com/sgl-project/sglang/pull/42557), [#45787](https://github.com/sgl-project/sglang/pull/45787)).\r\n* SGLang now runs on Google TPU7x through [SGLang-TorchTPU](https://github.com/sgl-project/sglang-torchtpu), a PyTorch-native plugin for TPU.\r\n* You can now serve LoRA adapters on quantized MoE models (FP8 and NVFP4) with a new opt-in LoRA engine ([#45892](https://github.com/sgl-project/sglang/pull/45892)).\r\n* DeepSeek-style MLA models running decode context parallelism can now keep their prefix cache in HiCache file-based L3 storage ([#41187](https://github.com/sgl-project/sglang/pull/41187)).\r\n\r\n## To upgrade\r\n\r\n```bash\r\nuv pip install --prerelease=allow sglang==0.5.23\r\n```\r\n\r\n| Platform | Docker |\r\n|---|---|\r\n| NVIDIA (CUDA 13) | `docker pull lmsysorg/sglang:v0.5.23` |\r\n| AMD MI35x | `docker pull lmsysorg/sglang:v0.5.23-rocm10-mi35x` |\r\n| AMD MI30x | `docker pull lmsysorg/sglang:v0.5.23-rocm10-mi30x` |\r\n| Intel GPU | `docker pull lmsysorg/sglang:v0.5.23-xpu` |\r\n| Intel CPU | `docker pull lmsysorg/sglang:v0.5.23-xeon` |\r\n| Google TPU7x (TorchTPU) | `docker pull lmsysorg/sglang:v0.1.0-tpu` |\r\n\r\n# Full Release Notes\r\n\r\n## Speculative Decoding\r\n\r\n* [Feat] speculative: Seed rejection-sampling draft proposals for deterministic inference: [#33395](https://github.com/sgl-project/sglang/pull/33395)\r\n* [Fix] observability: Correct speculative FPM timing and decode lengths: [#38711](https://github.com/sgl-project/sglang/pull/38711)\r\n* [Perf] spec: Let the FlashAttention backend capture the draft-extend CUDA graph (about 1.5x decode throughput at batch size 1 with a single-layer MoE drafter): [","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}