{"full_name":"nvidia-isaac/video_to_data","owner":"nvidia-isaac","name":"video_to_data","shard":"1","url":"https://github.com/nvidia-isaac/video_to_data","description":"Nvidia Isaac Video to Data Pipeline ","homepage":"","topics":[],"tag_pages":[],"language":"Python","license":"NOASSERTION","avatar":"https://avatars.githubusercontent.com/u/44278232?v=4","created_at":"2026-02-10T01:29:38Z","pushed_at":"2026-11-12T16:14:38Z","metrics":{"stars":1003,"forks":107,"watchers":27,"issues":7,"prs":215,"releases":4,"issues_open":1,"issues_closed":6,"prs_open":7,"prs_closed":30,"prs_merged":178,"discussions":0},"growth":{"day":{"stars":7,"forks":1,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-15","2026-11-16"],"stars":[996,1003],"forks":[106,107],"watchers":[25,27],"issues_open":[1,1],"prs_open":[7,7]},"boards":{"daily_watchers":192,"debut":31},"same_owner":[],"issues":[{"number":187,"title":"Question: Vega retarget reports zero converged frames for v0.4.0 reconstructed and GT-control inputs — how should this gate be interpreted?","comments":1,"created_at":"2026-09-30T04:01:34Z"}],"discussions":[],"latest_release":{"name":"Release v0.4.0","tag":"v0.4.0","published_at":"2026-09-23T22:02:16Z","body":"## Highlights\r\n\r\nV2D v0.4.0 introduces FlashCHORD for accelerated GPU-parallel robotic reference-tracking training for loco-manipulation and dexterous manipulation, alongside improvements to reconstruction tools, robotic grounding, and closed-loop GR00T evaluation.\r\n\r\n## FlashCHORD (NEW!)\r\n\r\nAccelerated policy training from human-object motion using Newton, Warp, and JAX.\r\n\r\n### Summary\r\n\r\n- Training recipes: adds FlashSAC and PPO training in Newton for floating Sharpa, Vega Sharpa, and G1 + Dex3.\r\n\r\n- Complete pipeline: documented setup, robot and scene inspection, kinematic and dynamic reference replay, training, evaluation, and policy viewer.\r\n\r\n- Motions and scenes: consumes the same parquet format as already released robotic grounding, existing motion references can directly be used with FlashCHORD.\r\n\r\n- Configuration: documents rewards, terminations, tracking tolerances, resets, curriculum, randomization, and debug-marker controls.\r\n\r\n- Execution: provides a dedicated Docker environment, local training progress, and OSMO templates for training and evaluation.\r\n\r\n### Known assumptions\r\n\r\n- The documented full training recipes use a 48 GiB GPU with at least 46,000 MiB free, Linux, NVIDIA driver 580 or newer, Docker, NVIDIA Container Toolkit, and Git LFS.\r\n\r\n### Known limitations\r\n\r\n- FlashCHORD uses its own container and training entrypoints. The existing robotic grounding and end-to-end GR00T workflow continues to use Isaac Lab. Support for Isaac Lab 3.0 with Newton backend will come in later releases.\r\n\r\nDocs: [robotic_grounding/flash_chord/README.md](https://github.com/nvidia-isaac/video_to_data/blob/main/robotic_grounding/flash_chord/README.md)\r\n\r\n## Robotic Grounding and End-to-End Workflow\r\n\r\n### Summary\r\n\r\n- Motion quality: adds signed-distance-field penetration checks for concave and hollow objects, with convex-hull fallback when no cached field is available.\r\n\r\n- Parallel evaluation: uses tiled wrist cameras for Vega Sharpa.\r\n\r\n## Reconstruction\r\n\r\n### Summary\r\n\r\n- Texture editing: adds an optional logo-redaction workflow with model-proposed regions, manual review, and synchronized GLB/PNG texture updates that preserve mesh geometry and UVs.\r\n\r\n- Mesh quality: adds input-file hashes, aligned dimensions, and surface-based scale estimates to geometry comparison reports.\r\n\r\n### Known limitations\r\n\r\n- Ego reconstruction (DynHaMeR, HaMeR, and HaMeR_mesh) may exhibit inconsistent object scale, orientation flips, and hand–object misalignment, particularly for nearly symmetric objects or automatically generated meshes. Results may vary between runs due to non-deterministic camera trajectory estimation and mesh generation.\r\n\r\n- CARI4D preserves the scale of the supplied object mesh. Meshes generated with SAM3D image-to-mesh must be scaled before being passed as --object_mesh_path; CARI4D does not automatically apply the separately estimated SAM3D transform.\r\n\r\nDocs: [object reconstruction](https://github.com/nvidia-isaac/video_to_data/blob/main/reconstruction/modules/v2d_hoi_object_reconstruction/README.md) · [CARI4D](https://github.com/nvidia-isaac/video_to_data/blob/main/reconstruction/modules/v2d_cari4d/README.md) · [egocentric reconstruction](https://github.com/nvidia-isaac/video_to_data/blob/main/reconstruction/docs/ego_e2e_setup.md)\r\n\r\n","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}