{"full_name":"NVIDIA/garak","owner":"NVIDIA","name":"garak","shard":"1","url":"https://github.com/NVIDIA/garak","description":"the LLM vulnerability scanner","homepage":"https://discord.gg/uVch4puUCs","topics":["ai","llm-evaluation","llm-security","security-scanners","vulnerability-assessment"],"tag_pages":["ai","llm-evaluation","llm-security","vulnerability-assessment"],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/1728152?v=4","created_at":"2023-05-10T18:52:16Z","pushed_at":"2026-11-06T23:26:29Z","metrics":{"stars":10037,"forks":1481,"watchers":66,"issues":1017,"prs":1355,"releases":34,"issues_open":308,"issues_closed":709,"prs_open":276,"prs_closed":246,"prs_merged":833,"discussions":74},"growth":{"day":{"stars":16,"forks":9,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[9979,9991,9997,10005,10011,10021,10037],"forks":[1464,1467,1467,1467,1469,1472,1481],"watchers":[null,66,66,66,66,66,66],"issues_open":[null,300,301,304,306,308,308],"prs_open":[null,268,268,270,274,274,276]},"boards":{},"same_owner":[{"full_name":"NVIDIA/DALI","name":"DALI","shard":"1","boards":{}},{"full_name":"NVIDIA/Isaac-GR00T","name":"Isaac-GR00T","shard":"1","boards":{}},{"full_name":"NVIDIA/Megatron-LM","name":"Megatron-LM","shard":"1","boards":{}},{"full_name":"NVIDIA/Model-Optimizer","name":"Model-Optimizer","shard":"1","boards":{}},{"full_name":"NVIDIA/NemoClaw","name":"NemoClaw","shard":"1","boards":{}},{"full_name":"NVIDIA/OpenShell","name":"OpenShell","shard":"1","boards":{"monthly":107,"monthly_rate":96}},{"full_name":"NVIDIA/SkillEvaluator","name":"SkillEvaluator","shard":"1","boards":{"daily":18,"weekly":138,"daily_rate":4,"weekly_rate":3,"monthly_rate":21,"daily_forks":18,"daily_watchers":189}},{"full_name":"NVIDIA/SkillSpector","name":"SkillSpector","shard":"1","boards":{"daily":148,"weekly":170,"monthly":91,"monthly_rate":110}},{"full_name":"NVIDIA/TensorRT-LLM","name":"TensorRT-LLM","shard":"1","boards":{}},{"full_name":"NVIDIA/apex","name":"apex","shard":"1","boards":{}},{"full_name":"NVIDIA/cosmos","name":"cosmos","shard":"1","boards":{}},{"full_name":"NVIDIA/cuda-samples","name":"cuda-samples","shard":"1","boards":{}},{"full_name":"NVIDIA/cudf","name":"cudf","shard":"1","boards":{}},{"full_name":"NVIDIA/cudnn-frontend","name":"cudnn-frontend","shard":"1","boards":{}},{"full_name":"NVIDIA/cutlass","name":"cutlass","shard":"1","boards":{}},{"full_name":"NVIDIA/nvidia-docker","name":"nvidia-docker","shard":"1","boards":{}},{"full_name":"NVIDIA/open-gpu-kernel-modules","name":"open-gpu-kernel-modules","shard":"1","boards":{}}],"issues":[{"number":1666,"title":"Proposal: Prompt defense posture detectors — assess system prompt hardening","comments":18,"created_at":"2026-04-03T15:04:13Z"},{"number":2136,"title":"judge detectors: unparseable judge verdict is scored as a successful attack","comments":10,"created_at":"2026-08-31T22:14:12Z"},{"number":579,"title":"probe: separation","comments":10,"created_at":"2024-04-02T11:15:19Z"},{"number":2445,"title":"probe: retrieval access-control leaks in RAG assistants, using planted canary documents","comments":8,"created_at":"2026-11-08T03:29:29Z"},{"number":2307,"title":"promptinject: rogue string is translated in the prompt but not in `triggers`, so non-English runs under-report hits","comments":8,"created_at":"2026-10-13T13:36:00Z"},{"number":844,"title":"migrate away from conda","comments":8,"created_at":"2024-08-21T05:38:32Z"},{"number":515,"title":"probe: aart","comments":8,"created_at":"2024-02-27T11:51:45Z"},{"number":2459,"title":"test_tox_unsafe and test_tox_safe assert nothing: the prompt has no lang, so the detectors return no scores","comments":7,"created_at":"2026-11-12T17:52:14Z"}],"discussions":[{"number":2319,"title":"v0.17.1","comments":0,"category":"Announcements","created_at":"2026-10-16T20:14:18Z"},{"number":2308,"title":"Runs with zero prompts report SKIP 0/0 without showing the provider error.\"","comments":3,"category":"General","created_at":"2026-10-13T22:14:59Z"},{"number":2237,"title":"Probe idea: output-contract adherence as a measurable property, separate from refusal","comments":0,"category":"Ideas and suggestions","created_at":"2026-09-26T07:17:11Z"},{"number":2215,"title":"Free-tier Groq rate limits cause multi-hour runtime with no visible warning (parallel_attempts + low TPM)","comments":5,"category":"Q&A","created_at":"2026-09-18T22:23:28Z"},{"number":2203,"title":"New contributor interested in LLM security probes and testing","comments":1,"category":"General","created_at":"2026-09-17T17:04:35Z"},{"number":2171,"title":"v0.17.0","comments":0,"category":"Announcements","created_at":"2026-09-09T18:56:03Z"},{"number":2119,"title":"Benchmarked garak against two third-party targets, corrections welcome","comments":0,"category":"Q&A","created_at":"2026-08-26T09:14:39Z"},{"number":2116,"title":"finding-bridge: a hitlog consumer that seals hits for triage","comments":0,"category":"General","created_at":"2026-08-25T12:58:06Z"}],"latest_release":{"name":"v0.17.1","tag":"v0.17.1","published_at":"2026-10-16T20:14:17Z","body":"## What's Changed\r\n\r\n### New features\r\n* feat(evaluators): add hit@k attack success rate metric by @stefanoamorelli in https://github.com/NVIDIA/garak/pull/2090\r\n  This new metric is currently exposed as terminal output and available per probe/detector pair in the final digest entry of the report.jsonl\r\n* Add Python 3.14 support by @NK-Works in https://github.com/NVIDIA/garak/pull/2297\r\n\r\n### Improved plugins\r\n* fix(detectors): match mitigation.Prefixes at the start of the response by @chiruu12 in https://github.com/NVIDIA/garak/pull/2067\r\n* fix(detectors): stop FileDetector scoring an unexamined file as a pass by @CTWalk in https://github.com/NVIDIA/garak/pull/2080\r\n* fix(detectors): stop RefusalOnlyAdversarial over-counting skipped multi-turn attempts by @Yigtwxx in https://github.com/NVIDIA/garak/pull/2111\r\n* fix(detectors): scope safe-token suppression to its own match by @feiiiiii5 in https://github.com/NVIDIA/garak/pull/2182\r\n* fix(detectors): Scope ApiKey safe-token suppression to the matching credential by @MRX-72 in https://github.com/NVIDIA/garak/pull/2270\r\n* fix(detectors): Do not treat octal 27 as a raw ANSI escape by @SashaMIT in https://github.com/NVIDIA/garak/pull/2255\r\n* fix(probes): keep promptinject settings aligned with sampled prompts by @reeve25 in https://github.com/NVIDIA/garak/pull/2260\r\n* fix(probes): restore DAN prompt compatibility by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/2282\r\n\r\n### Documentation\r\n* docs(rest): document updated CLI syntax and generator_options schema by @NK-Works in https://github.com/NVIDIA/garak/pull/2279\r\n\r\n### Tuning & fixes\r\n* fix(generators): raise clear error on HF InferenceAPI 404 response by @pujitha24 in https://github.com/NVIDIA/garak/pull/2112\r\n* fix(buffs): use prompt setter in paraphrase buffs to update conversations by @bodapatisaikrishna in https://github.com/NVIDIA/garak/pull/2200\r\n* fix: deep-copy mutable attempt state in Buff._derive_new_attempt by @pujitha24 in https://github.com/NVIDIA/garak/pull/2156\r\n* fix(buffs): preserve attempt attributes and isolate mutable state in _derive_new_attempt by @bodapatisaikrishna in https://github.com/NVIDIA/garak/pull/2198\r\n* fix: reuse the WebSocket event loop across calls by @Vamsi-2581 in https://github.com/NVIDIA/garak/pull/2269\r\n* fix: respect parallel_capable for generator requests by @denelbjohn18 in https://github.com/NVIDIA/garak/pull/2274\r\n* probes: factor up local prompt-cap logic into probes.base by @VishnuR23 in https://github.com/NVIDIA/garak/pull/2085\r\n* task: update deps for security hygiene by @jmartin-tech in https://github.com/NVIDIA/garak/pull/2311\r\n* lock transformers due to interface changes by @jmartin-tech in https://github.com/NVIDIA/garak/pull/2316\r\n* Use --target_name as the WebSocket generator uri by @Somay-kousis in https://github.com/NVIDIA/garak/pull/2315\r\n* report: update report probe tooltip text to be accurate (higher=better) by @leondz in https://github.com/NVIDIA/garak/pull/2317\r\n\r\n## New Contributors\r\n* @bodapatisaikrishna made their first contribution in https://github.com/NVIDIA/garak/pull/2200\r\n* @pujitha24 made their first contribution in https://github.com/NVIDIA/garak/pull/2112\r\n* @Vamsi-2581 made their first contribution in https://github.com/NVIDIA/garak/pull/2269\r\n* @denelbjohn18 made their first contribution in https://github.com/NVIDIA/garak/pull/2274\r\n* @MRX-72 made their first contribution in https://github.com/NVIDIA/garak/pull/2270\r\n* @reeve25 made their first contribution in https://github.com/NVIDIA/garak/pull/2260\r\n* @VishnuR23 made their first contribution in https://github.com/NVIDIA/garak/pull/2085\r\n* @SashaMIT made their first contribution in https://github.com/NVIDIA/garak/pull/2255\r\n* @NK-Works made their first contribution in https://github.com/NVIDIA/garak/pull/2297\r\n* @chiruu12 made their first contribution in https://github.com/NVIDIA/garak/pull/2067\r\n* @CTWalk made their first contribution in https://github.com/NVIDIA/garak/pull/2080\r\n* @Yi","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}