Integrating a physics-based engine with the SceneMaker model will enhance the realism and functional utility of generated 3D scenes, especially in dynamic environments.

Motivation

While SceneMaker improves geometric quality and pose accuracy, it lacks a consideration for physical interactions within the generated scenes, which are crucial for applications in robotics and VR/AR. By incorporating a physics engine, the generated scenes can be made more realistic and applicable in simulations where physical interactions are pivotal.

Proposed Method

Develop an integrated system that combines SceneMaker with a physics engine like Bullet or PhysX. Test the system by generating scenes that involve dynamic interactions (e.g., falling objects, collisions) and evaluate the realism and accuracy of these interactions compared to real-world scenarios. Conduct user studies in VR environments to assess perceived realism.

Expected Contribution

This research would demonstrate the feasibility of generating 3D scenes that not only look realistic but also behave realistically, expanding the scope of applications in simulation-driven fields.

Required Resources

Access to a physics engine, computational resources for running simulations, expertise in physics-based modeling, and a VR setup for user testing.

Source Paper

SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model

View Paper Details →