{"full_name":"NVIDIA-NeMo/Speech","owner":"NVIDIA-NeMo","name":"Speech","shard":"1","url":"https://github.com/NVIDIA-NeMo/Speech","description":"A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)","homepage":"https://docs.nvidia.com/nemo/speech/nightly/index.html","topics":["machine-translation","speaker-recognition","asr","tts","generative-ai","deeplearning","neural-networks","speaker-diariazation","speech-translation","speech-synthesis"],"tag_pages":["asr","deeplearning","generative-ai","machine-translation","neural-networks","speech-synthesis","tts"],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/213689629?v=4","created_at":"2019-08-05T20:16:42Z","pushed_at":"2026-11-15T11:33:16Z","metrics":{"stars":18754,"forks":3693,"watchers":230,"issues":3089,"prs":12688,"releases":87,"issues_open":145,"issues_closed":2944,"prs_open":323,"prs_closed":3033,"prs_merged":9332,"discussions":696},"growth":{"day":{"stars":3,"forks":2,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[18722,18734,18740,18742,18747,18751,18754],"forks":[3686,3687,3689,3690,3691,3691,3693],"watchers":[null,230,230,230,230,230,230],"issues_open":[null,143,143,143,145,145,145],"prs_open":[null,304,305,305,330,322,323]},"boards":{},"same_owner":[{"full_name":"NVIDIA-NeMo/Megatron-Bridge","name":"Megatron-Bridge","shard":"1","boards":{}}],"issues":[{"number":13825,"title":"Poor WER when trying to fine-tune Parakeet v2 TDT  to other dataset than English","comments":54,"created_at":"2025-06-04T14:31:49Z"},{"number":15037,"title":"Intermittent CUDA Out of Memory Errors during Parakeet-TDT-0.6B Fine-tuning","comments":27,"created_at":"2025-11-06T07:50:38Z"},{"number":14140,"title":"TDT Head Stagnation in parakeet-tdt_ctc-110m Fine-tuning on Persian Data","comments":26,"created_at":"2025-07-06T16:38:51Z"},{"number":14613,"title":"SOS! Nemo:25.07 Container Image Cmpatibility mode is UNAVAILABLE ERROR!","comments":21,"created_at":"2025-08-29T20:38:23Z"},{"number":15820,"title":"nvidia/nemotron-3.5-asr-streaming-0.6b: transcribe() fails with \"ValueError: Unknown prompt key: 'None'\"","comments":10,"created_at":"2026-06-22T13:04:33Z"},{"number":14427,"title":"compute_rnnt_timestamps fails with empty char offsets","comments":10,"created_at":"2025-08-07T14:31:44Z"},{"number":15615,"title":"Crazy Logging I want to shut it down","comments":9,"created_at":"2026-04-16T10:17:06Z"},{"number":15514,"title":"Question: Canary-v2 hallucination in streaming mode","comments":8,"created_at":"2026-03-18T10:23:50Z"}],"discussions":[{"number":16460,"title":"Report: single-speaker Parakeet TDT 0.6B v2 fine-tune for tech dictation (Fritter)","comments":0,"category":"Public Models","created_at":"2026-10-27T16:25:39Z"},{"number":16404,"title":"Norwegian support for lightweight Parakeet models","comments":0,"category":"General","created_at":"2026-10-19T07:34:50Z"},{"number":16163,"title":"Is GPU-PB phrase boosting usable on Apple Silicon (MPS / CPU-only), or is it effectively NVIDIA-only?","comments":1,"category":"Q&A","created_at":"2026-08-29T03:10:01Z"},{"number":16116,"title":"Built double-tap-Ctrl dictation for Linux on nemotron-speech-streaming-en-0.6b","comments":0,"category":"Show and tell","created_at":"2026-08-20T01:50:23Z"},{"number":15900,"title":"Nemotron 3.5 ASR Streaming 0.6B in native C++/GGML. RTF = 0.0066 on RTX 5090","comments":0,"category":"General","created_at":"2026-07-09T03:14:59Z"},{"number":15731,"title":"What If NeMo Had Actual Physics?","comments":0,"category":"Ideas","created_at":"2026-05-26T22:54:56Z"},{"number":15667,"title":"Nemo Toolkit is not in nvcr.io/nvidia/nemo:26.04.00?","comments":1,"category":"Q&A","created_at":"2026-05-06T19:19:38Z"},{"number":15513,"title":"About the NeMo Framework restructuring","comments":1,"category":"Q&A","created_at":"2026-03-18T09:03:12Z"}],"latest_release":{"name":"NVIDIA NeMo Speech 3.0","tag":"v3.0.0","published_at":"2026-08-07T00:13:21Z","body":"# NVIDIA NeMo Speech 3.0 Release Notes\r\n\r\nNeMo Speech 3.0 is the first major release after the repo split and rename to `NVIDIA-NeMo/Speech`. The repo now focuses on ASR, TTS, audio processing, speaker tasks, and SpeechLM. Non-speech Framework, LLM, VLM, diffusion, export/deploy, evaluator components can now be found under separate repos under NVIDIA-NeMo organization.\r\n\r\nWhile this release brings new major features, the central focus is addressing technical debt: removed 800k deprecated LOC, migration to uv package manager for cleaner installs, stronger model test coverage, reduced the number of dependencies, revamped documentation, lighter containers, and `AGENTS.md` + agentic skills. \r\n\r\n## Top-Level Changes and Highlights\r\n\r\n### Breaking Changes and Migration Notes\r\n\r\n- LLM/VLM/diffusion/NLP/export/evaluator collections have been moved out to separate repos.\r\n- Unsupported ASR/TTS models and old tutorials were removed. Use active examples under `examples/` and docs under `docs/source/`.\r\n- `uv sync` reproduces the supported NeMo Speech stack and may replace Python/PyTorch/CUDA inside `.venv`. To keep your own stack, install PyTorch first, then use `uv pip` or `pip`.\r\n- SpeechLM2 compiled acceleration is optional. Use the supported Docker build for TE/FlashAttention/Mamba/grouped-GEMM/DeepEP performance paths.\r\n\r\n### SpeechLM2 and NeMo Automodel Backend\r\n\r\nSpeechLM2 now supports training SpeechLM models with NeMo Automodel through `SALMAutomodel`, targeting both dense and MoE LLM backbones such as Nemotron 3.\r\n\r\n- `SALMAutomodel` adds an Automodel-backed SALM path with native LoRA, deferred `configure_model()` initialization, shard-aware distributed loading, and Automodel-owned mesh creation (#15447).\r\n- `AutomodelParallelStrategy` supports FSDP2, HSDP, TP, CP, and EP, letting SpeechLM training combine sharded data parallelism, tensor/model partitioning, long-context sequence sharding, and MoE expert routing (#15447, #15648, #15679, #15773).\r\n- Optimized backend support now covers Transformer Engine, FlashAttention, Mamba/state-space kernels, grouped GEMM for MoE experts, and DeepEP for expert-parallel all-to-all communication (#15737, #15758, #15679).\r\n- THD packed-sequence training reduces padding waste in variable-length speech batches and is the supported path for CP long-audio training (#15679).\r\n- Long-audio support was expanded with encoder chunking, activation-checkpointing controls, CP-safe data handling, and fixes around dtype/checkpoint restoration (#15648, #15679, #15686, #15716).\r\n- Export and serving improved with vLLM integration, HF/vLLM-ready checkpoint export, backbone-native chat templates, offline HF export, and buffered SALM inference (#15364, #15520, #15623, #15736).\r\n- Data format support now includes ShareGPT conversations, indexed ShareGPT JSONL, and WebDataset-backed ShareGPT conversations (#15316, #15410).\r\n\r\n### Nemotron VoiceChat Training Modules\r\n\r\nNeMo Speech 3.0 adds the train/eval modules behind Nemotron VoiceChat.\r\n\r\n- Added Nemotron VoiceChat speech-decoder modules for the response-generation side of the duplex stack (#15066).\r\n- Added DuplexSTT training and inference for generating agent text from user speech plus text context (#15092).\r\n- Added Duplex/EAR-TTS speech-generation pieces used by VoiceChat, including MagpieTTS decoder integration and semantic-codec training support (#15277, #15524).\r\n- Added the joint `NemotronVoiceChat` STT + TTS class for validation, offline speech-to-speech inference, and export workflows (#15456).\r\n- Added formatter/reproducibility work for the VoiceChat speech decoder, with faster training and half-precision inference support (#15583).\r\n- Expanded Voice Agent examples for Nemotron Nano v2/v3, MagpieTTS, tool calling, audio logging, and README guidance (#14325, #14704, #15243, #15269, #15318, #15547, #15642).\r\n- Hardened Voice Agent behavior around default Parakeet EOU STT, text aggregation, end-of-bot handling, RTVI messages, empty tools, and logging (#1","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}