{"full_name":"Neywa-Labs/mlx-vlm","owner":"Neywa-Labs","name":"mlx-vlm","shard":"1","url":"https://github.com/Neywa-Labs/mlx-vlm","description":"MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.","homepage":"https://neywa-labs.github.io/mlx-vlm/","topics":["llava","llm","mlx","vision-transformer","apple-silicon","idefics","local-ai","paligemma","vision-framework","vision-language-model","florence2","molmo","pixtral"],"tag_pages":["apple-silicon","llm","local-ai","mlx","vision-language-model","vision-transformer"],"language":"Python","license":"MIT","avatar":"https://avatars.githubusercontent.com/u/228382334?v=4","created_at":"2024-04-16T15:10:12Z","pushed_at":"2026-11-15T16:02:48Z","metrics":{"stars":5848,"forks":893,"watchers":35,"issues":929,"prs":1750,"releases":93,"issues_open":141,"issues_closed":788,"prs_open":135,"prs_closed":417,"prs_merged":1198,"discussions":22},"growth":{"day":{"stars":2,"forks":1,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[5830,5834,5840,5840,5845,5846,5848],"forks":[886,887,888,888,891,892,893],"watchers":[null,35,35,35,35,35,35],"issues_open":[null,132,135,134,134,139,141],"prs_open":[null,129,133,133,130,134,135]},"boards":{},"same_owner":[],"issues":[{"number":39,"title":"Models to port to MLX-VLM","comments":74,"created_at":"2024-06-11T12:10:59Z"},{"number":999,"title":"Server clears Metal cache after every request, destroying KV prefix cache and forcing full re-prefill on every turn","comments":7,"created_at":"2026-04-09T05:37:50Z"},{"number":286,"title":"Does MLX-VLM support fine-tuning of vision tower layers?","comments":7,"created_at":"2025-04-04T12:08:53Z"},{"number":1781,"title":"Server: add backend-aware hotword and custom vocabulary support for audio transcriptions","comments":6,"created_at":"2026-08-03T00:34:49Z"},{"number":2254,"title":"glm5_next: remaining KDA decode kernel-launch overhead after #2127 (spec + measurements for a fused/compiled step)","comments":5,"created_at":"2026-09-14T19:26:33Z"},{"number":1925,"title":"Qwen3.8-MLX vision NOT work with base64 data-URI","comments":5,"created_at":"2026-08-16T18:49:57Z"},{"number":1629,"title":"Redesign APC around pluggable cache components for custom attention and hybrid models","comments":5,"created_at":"2026-07-18T10:22:39Z"},{"number":1562,"title":"APC + kv-bits: staggered continuous-batching join crashes (last-layer type mismatch)","comments":5,"created_at":"2026-07-10T20:57:34Z"}],"discussions":[{"number":2456,"title":"AREX-2 (Qwen3.8-27B fine-tune) converted with 0.7.4: works well at 5/6/8-bit, plain 4-bit loops at temperature 0","comments":1,"category":"Show and tell","created_at":"2026-10-07T14:25:28Z"},{"number":2165,"title":"SmolVLA on Apple Silicon: an MLX runtime using mlx-vlm’s SmolVLM as an architecture reference","comments":1,"category":"Show and tell","created_at":"2026-09-05T12:26:33Z"},{"number":2026,"title":"EVIE-Preview-4.5B on Apple Silicon: MLX Q4 visual retrieval","comments":0,"category":"Show and tell","created_at":"2026-08-26T09:46:41Z"},{"number":1364,"title":"MCP to help OSS devs understand Blaizzy/mlx-vlm","comments":0,"category":"General","created_at":"2026-06-12T12:07:35Z"},{"number":1290,"title":"KVarN: 3-5x more context, throughput above FP16, and FP16-level accuracy. from Huawei is used to back their Qwen workload. would be nice to add here","comments":0,"category":"Ideas","created_at":"2026-06-04T16:30:21Z"},{"number":1258,"title":"v0.6.0","comments":1,"category":"Announcements","created_at":"2026-06-01T19:57:42Z"},{"number":1025,"title":"Does mlx vlm respect to transformers generation config when generating response?","comments":0,"category":"Q&A","created_at":"2026-04-15T15:47:32Z"},{"number":932,"title":"Memory issues when fine-tuning Qwen2-VL with LoRA on 16GB M1 Max - any optimization tips?","comments":2,"category":"Q&A","created_at":"2026-04-05T08:06:20Z"}],"latest_release":{"name":"v0.7.12","tag":"v0.7.12","published_at":"2026-11-04T17:03:29Z","body":"## What's Changed\r\n* Lower MOSS-Image peak memory during VAE decode by @felix-ab in https://github.com/Neywa-Labs/mlx-vlm/pull/2615\r\n* Fix LFM3 thinking leakage by initializing response parsers with the prompt by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2621\r\n* Add Raon-1 Mini support by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2622\r\n* Support text-only ASR with diarized_json by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2626\r\n* Send SSE heartbeats during idle OpenAI streams by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2633\r\n* Add native Mage-VL-Cowork support by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2636\r\n* Support speculative thinking budgets and expose reasoning efforts by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2641\r\n* Add Mistral Large 4 by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2629\r\n* fix: accept top_k equal to vocab size in apply_top_k by @ShryukGrandhi in https://github.com/Neywa-Labs/mlx-vlm/pull/2643\r\n* Make the LFM3-VL processor follow the transformers processor contract by @abhijithneilabraham in https://github.com/Neywa-Labs/mlx-vlm/pull/2640\r\n* Expose the server's reasoning-effort control in the generate CLI by @selimfedakar in https://github.com/Neywa-Labs/mlx-vlm/pull/2631\r\n* Filter per-row M-RoPE deltas when the speculative batch drops finished rows by @selimfedakar in https://github.com/Neywa-Labs/mlx-vlm/pull/2625\r\n* Keep tool-call streaming alive on outputs the final parser tolerates by @selimfedakar in https://github.com/Neywa-Labs/mlx-vlm/pull/2635\r\n* convert: keep the source tokenizer.json after save_pretrained by @JarJarFran in https://github.com/Neywa-Labs/mlx-vlm/pull/2624\r\n* fix(flint): view flat uint8 quantized weights as packed uint32 by @martinstiago in https://github.com/Neywa-Labs/mlx-vlm/pull/2616\r\n* Fix Gemma 4 tool calls containing braces in strings by @victor-torres in https://github.com/Neywa-Labs/mlx-vlm/pull/2593\r\n* fix: reload native LFM2 ColBERT checkpoints by @kocean9-freedom in https://github.com/Neywa-Labs/mlx-vlm/pull/2575\r\n* fix(tool-parsers): keep the text \"null\" for string-typed qwen3_coder parameters by @parasew in https://github.com/Neywa-Labs/mlx-vlm/pull/2588\r\n* Preserve inference state when resuming interrupted responses by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2623\r\n* Honor APC memory admission and keep warm requests decoding by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2645\r\n* Reduce APC merge memory and preserve custom checkpoints by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2646\r\n* Fix typical-p filtering with masked tokens by @HowieYin in https://github.com/Neywa-Labs/mlx-vlm/pull/2563\r\n* fix: honor base64 embedding response encoding by @kocean9-freedom in https://github.com/Neywa-Labs/mlx-vlm/pull/2573\r\n* Build the Qwen3 text reranker prompt without the chat template by @begcdn in https://github.com/Neywa-Labs/mlx-vlm/pull/2561\r\n* fix: antialias position embedding resizes with PyTorch's triangle filter by @rrrxxx0510 in https://github.com/Neywa-Labs/mlx-vlm/pull/2553\r\n* Honor configured activations in BERT and XLM-R encoders by @kocean9-freedom in https://github.com/Neywa-Labs/mlx-vlm/pull/2572\r\n* Expose continuation idle timeout to clients by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2652\r\n* Reduce continuation idle timeout to two minutes by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2654\r\n* Add native Idol image-to-3D generation by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2644\r\n* fix(qwen3_vl): compute text-only rope deltas from unpadded length by @pujitha24 in https://github.com/Neywa-Labs/mlx-vlm/pull/2587\r\n* Keep resumed DFlash and MTP requests in shared speculative batches by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2648\r\n* fix: compute text-only rope deltas from unpadded length in the other MRoPE models by @ahuzmeza in https://github.com/Neywa-Labs/mlx-vlm/pull/2658\r\n","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}