Robot deployment RoboBrain2.0 inference strategy

[LongXinKou]

Hello, thank you for your great work on RoboBrain2.0. I have two quick questions regarding deployment and inference strategy:

• How can RoboBrain2.0 be deployed onto a robot or a simulator? Is there any existing interface or recommended pipeline for integrating it with real or simulated robot control?
• Does RoboBrain2.0 interact with the environment by predicting keypoints or trajectory as part of its inference process? Or does it rely on another type of representation for robot actuation?

Thank you in advance for your clarification!

[Yuheng2000]

RoboOS is a supporting framework developed by our team — you’re welcome to try it out!
For more details and to get started, please visit the official repository: https://github.com/FlagOpen/RoboOS.

Within this framework, RoboBrain can interact with robots in multiple ways. It can invoke various action models (e.g., AnyGrasp for robotic grasping) via tool calling, or output 2D keypoints that are then projected into 3D coordinates using the robot’s depth camera intrinsics. These 3D points are further transformed into world coordinates through the camera-to-robot extrinsic matrix, and the robot’s inverse kinematics (IK) module is then used to reach the corresponding position.