{"full_name":"huggingface/accelerate","owner":"huggingface","name":"accelerate","shard":"1","url":"https://github.com/huggingface/accelerate","description":"🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support","homepage":"https://huggingface.co/docs/accelerate","topics":[],"tag_pages":[],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/25720743?v=4","created_at":"2020-10-30T13:27:12Z","pushed_at":"2026-11-14T00:44:13Z","metrics":{"stars":9947,"forks":1607,"watchers":94,"issues":2060,"prs":2463,"releases":73,"issues_open":4,"issues_closed":2056,"prs_open":7,"prs_closed":579,"prs_merged":1877,"discussions":0},"growth":{"day":{"stars":3,"forks":-2,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[9937,9940,9941,9941,9942,9944,9947],"forks":[1597,1599,1602,1604,1607,1609,1607],"watchers":[null,94,94,94,94,94,94],"issues_open":[null,86,87,88,89,4,4],"prs_open":[null,101,104,109,114,4,7]},"boards":{},"same_owner":[{"full_name":"huggingface/transformers","name":"transformers","shard":"1","boards":{"total":54}},{"full_name":"huggingface/OpenEnv","name":"OpenEnv","shard":"1","boards":{}},{"full_name":"huggingface/agents-course","name":"agents-course","shard":"1","boards":{}},{"full_name":"huggingface/alignment-handbook","name":"alignment-handbook","shard":"1","boards":{}},{"full_name":"huggingface/diffusers","name":"diffusers","shard":"1","boards":{}},{"full_name":"huggingface/ml-intern","name":"ml-intern","shard":"1","boards":{}},{"full_name":"huggingface/parler-tts","name":"parler-tts","shard":"1","boards":{}},{"full_name":"huggingface/pytorch-image-models","name":"pytorch-image-models","shard":"1","boards":{}},{"full_name":"huggingface/sentence-transformers","name":"sentence-transformers","shard":"1","boards":{}},{"full_name":"huggingface/skills","name":"skills","shard":"1","boards":{}},{"full_name":"huggingface/smolagents","name":"smolagents","shard":"1","boards":{}},{"full_name":"huggingface/speech-to-speech","name":"speech-to-speech","shard":"1","boards":{}},{"full_name":"huggingface/transformers.js","name":"transformers.js","shard":"1","boards":{}},{"full_name":"huggingface/trl","name":"trl","shard":"1","boards":{}}],"issues":[{"number":1089,"title":"how to cleanly exit when using `accelerate launch`","comments":25,"created_at":"2023-02-16T13:41:23Z"},{"number":4555,"title":"Sequence parallelism: let an SP-aware model opt out of the linear-attention refusal, and make `disable_in_eval` configurable","comments":2,"created_at":"2026-11-13T23:18:49Z"},{"number":4525,"title":"CVE-2026-69112 / GHSA-4j2p-28q2-5m79: advisory says <= 1.14.0, but 1.15.0 has the same code. Please clarify the project's position.","comments":2,"created_at":"2026-11-11T21:27:16Z"},{"number":4553,"title":"FSDP2 `cpu_ram_efficient_loading`: ignored modules get sharded, and persistent buffers crash the load","comments":0,"created_at":"2026-11-13T23:14:22Z"}],"discussions":[],"latest_release":{"name":"v1.15.0: FSDP2 activation memory, dtensor improvements","tag":"v1.15.0","published_at":"2026-09-09T13:30:33Z","body":"# v1.15.0: FSDP2 activation memory, dtensor improvements\r\n\r\n## FSDP2\r\n\r\nA large batch of FSDP2 work this release: two fixes that cut activation memory at long sequence lengths, tied-embedding support on torch >= 2.13, and a round of checkpointing correctness and scale fixes.\r\n\r\nActivation checkpointing was wrapping each *child* of the matched layer (`self_attn`, `mlp`, the norms) instead of the layer itself, so every inter-child activation stayed saved for backward. It now wraps the layer. \r\n\r\nThere's also a new FSDP2-only `activation_checkpointing_offload`, which moves the remaining per-layer checkpoint inputs to pinned CPU memory. Gradients are exactly those of plain activation checkpointing: \r\n\r\n```yaml\r\n# fsdp2.yaml\r\nfsdp_config:\r\n  fsdp_version: 2\r\n  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP\r\n  fsdp_activation_checkpointing: true\r\n  fsdp_activation_checkpointing_offload: true\r\n```\r\n\r\n```bash\r\naccelerate launch --config_file fsdp2.yaml train.py\r\n```\r\n\r\n- FSDP2 activation checkpointing: wrap the matched transformer layer itself, not each of its children by @qgallouedec in #4172\r\n- Add FSDP2 `activation_checkpointing_offload`: offload checkpointed layer inputs to pinned CPU memory by @qgallouedec in #4175\r\n- Fix FSDP2 tied-embedding models on torch >= 2.13: put the output embedding in the same fully_shard group by @qgallouedec in #4171\r\n- Fix FSDP2/PEFT/`FULL_STATE_DICT` dropping every rank's adapter shard except rank 0 by @AmineDiro in #4206\r\n- FSDP2: per-rank torch.save/load for SHARDED_STATE_DICT to fix 2800+ NPU checkpoint timeout by @gygdh-001 in #4105\r\n- Fix FSDP sharded checkpoint path resolution by @HaomingSong in #4119\r\n- Raise a clear error when FSDP is enabled on a mesh with no shard dimension by @qgallouedec in #4180\r\n\r\n## DTensor\r\nTwo fixes for DTensor-sharded models, which you hit with FSDP2, tensor parallelism, or any N-D parallelism setup: gradient clipping no longer fails on the `foreach` op when plain tensors and DTensors are mixed, and `prepare_model` leaves an already-sharded model where it is:\r\n- Clip grad norm support for dtensors by @michaelbenayoun in #4219\r\n- prepare_model: don't move DTensor-sharded models to the device by @qgallouedec in #4181\r\n\r\n## Offloading & Quantization\r\n\r\nAn entire model can now be dispatched to disk, including tied weights — useful for tools like `llm-compressor` that compress large models on machines that can't hold them:\r\n\r\n- [Offloading] Support full disk offloading by @kylesayrs in #4083\r\n- Fix disk offload crash on FP8 tensors by @shoemoney in #4151\r\n\r\n## Trackers\r\n\r\nCustom trackers can be registered by name and then selected from `log_with=` like any built-in one:\r\n\r\n```python\r\nfrom accelerate import Accelerator\r\nfrom accelerate.tracking import register_tracker_class\r\n\r\nregister_tracker_class(MyTracker)  # MyTracker.name == \"my_tracker\"\r\naccelerator = Accelerator(log_with=\"my_tracker\")\r\n```\r\n\r\n- Add register_tracker_class to register custom trackers by name by @javierdejesusda in #4060\r\n- Fix MLFLOW_NESTED_RUN never being able to turn nesting off by @yupengtang in #4223\r\n\r\n## Device support\r\n\r\nNeuron gains a torch dynamo backend (so `--torch-compile` works with the Transformers Trainer) and MPS is now reported and handled properly by `accelerate env` and `find_executable_batch_size`.\r\n\r\n- Add `neuron` backend for torch dynamo by @michaelbenayoun in #4097\r\n- Add the neuron device branch in state by @michaelbenayoun in #4218\r\n- Report MPS as the accelerator in `accelerate env` by @xquantize in #4158\r\n- Fix accelerate env never reading the MPS chip name by @yupengtang in #4221\r\n- Treat MPS out-of-memory errors as OOM in find_executable_batch_size by @caiotheodoro in #4227\r\n- Fix memory accounting for integrated CUDA devices by @harshitakrishna18 in #4187\r\n- Ensure device-agnostic in docs by @kaixuanliu in #4163\r\n\r\n## CLI\r\n\r\n- Handle missing accelerate executable in env command by @tandede in #4168\r\n- Fix accelerate launch --cpu never setting the KMP variables by @yupe","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}