Under the Hood:
Intelligent 3D Scene Composition in One Click
An end-to-end pipeline to automate 3D scenes in seconds.
An end-to-end pipeline to automate 3D scenes in seconds.
Our early [ proof of concept ← ] came in the form of intelligent 3D scene composition in one click.
This end-to-end scene automation pipeline was achieved with a patchwork of vision language models, and was showcased [ above ↑ ] as an editor plugin within a 3D gaming engine (Unreal Engine).
The pipeline was two stage:
Asset ingestion and understanding
How can we store, visualise, and "embed" assets for 3D AI understanding and
downstream deployment?
Asset selection and positioning
How can we retrieve and deploy relevant assets "in context" (within the gaming
engine) and "in position" (with plausible locations within 3D scenes) from a
single in-game user prompt?
A high-level solution architecture is depicted below:
[ Click to expand ↑ ]