Using SceneMaker's de-occlusion module as a predictive prior for Next-Best-View (NBV) planning will achieve higher scene coverage rates in robotic exploration compared to standard frontier-based or volumetric methods.

Motivation

The paper's core innovation is effective de-occlusion (hallucinating hidden geometry). In robotics, exploration algorithms struggle to decide where to look next. By using SceneMaker to predict what lies behind occlusions, a robot can prioritize viewing angles that verify or refine these predictions, rather than exploring randomly.

Proposed Method

Deploy a mobile robot agent in a simulated environment (e.g., Habitat-Sim). Replace the standard mapping module with SceneMaker. At each step, generate the de-occluded geometry of the current view. Calculate an 'uncertainty map' based on the variance of the generative model's output. Direct the robot to the camera pose that maximally reduces this uncertainty. Compare exploration speed against standard frontier exploration.

Expected Contribution

Demonstration of generative AI acting as a functional component in active perception and robotic planning, proving transferability beyond static scene generation.

Required Resources

Robotics simulator (Habitat or Isaac Sim), reinforcement learning framework, pre-trained SceneMaker model.

Source Paper

SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model

View Paper Details →