Skip to content

Conversation

@YilingQiao
Copy link
Collaborator

@YilingQiao YilingQiao commented Jan 8, 2026

Summary

Adds support for cameras (both attached and static) to render different viewpoints per environment when n_envs > 1. Previously, all environments rendered from the same camera pose. Here are batched rendered results for rasterizer and raytracer

camera_multi_env camera_multi_env_rt

Implementation

Core Change: Added _get_camera_transforms_per_env(n_envs) method in camera.py that computes unique camera transforms for each environment:

  • For attached cameras: transforms local position with each env's link pose, computes proper lookat-based view
  • For static cameras: returns same transform for all envs

Rasterizer (rasterizer.py, pyrender/renderer.py):

  • Added camera_poses parameter to render pipeline
  • Updates camera view matrix per-env inside existing env loop
  • Uses instanced rendering - no performance penalty

Raytracer (raytracer.py):

  • Added update_scene_for_env(env_idx) method that updates shape _0 with transforms from specified env
  • Hides other env shapes by moving them off-screen
  • Renders sequentially (O(N) slower, documented)

Files Changed

  • genesis/engine/sensors/camera.py - Core camera transform logic
  • genesis/vis/rasterizer.py - Pass camera_poses to pyrender
  • genesis/ext/pyrender/renderer.py - Per-env camera matrix updates
  • genesis/vis/raytracer.py - Per-env scene updates

Testing

# Run all camera tests (10 tests)
pytest tests/test_sensor_camera.py -v --forked

# Run example
python examples/sensors/camera_multi_env.py -n 4  # Rasterizer
python examples/sensors/camera_multi_env.py -n 4 -r raytracer  # Raytracer

Test Coverage

  • test_rasterizer_camera_sensor_attached_per_env_pose - Attached camera different poses per env
  • test_raytracer_camera_sensor_attached_per_env_pose - Same for raytracer
  • test_*_different_poses - Static camera sees different object positions per env

Checklist:

  • I read the CONTRIBUTING document.
  • I followed the Submitting Code Changes section of CONTRIBUTING document.
  • I tagged the title correctly (including BUG FIX/FEATURE/MISC/BREAKING)
  • I updated the documentation accordingly or no change is needed.
  • I tested my changes and added instructions on how to test it for reviewers.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@YilingQiao YilingQiao changed the title Yiling/260107 multi env rasterizer sensor [FEATURE] Multi env camera sensor (pyrender and Luisa) Jan 8, 2026
@YilingQiao YilingQiao force-pushed the yiling/260107_multi_env_rasterizer_sensor branch from 2e3a639 to 832541f Compare January 8, 2026 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant