Project field note 2025
SimGen
A multimodal sketch-to-simulation prototype that turns drawings and text into interactive physics scenes.
Sketching with physics
SimGen began with a simple interaction: draw a physical system, describe what should happen, and turn the combination into a simulation you can manipulate.
The browser interface combines a sketch canvas with text input. A backend pipeline interprets the drawing and description, builds a structured representation of the scene, and passes it to an interactive runtime. The result is a prototype for moving from an imprecise human idea to an environment with objects, constraints, motion, and collisions.
Why this direction
Physics tools are usually precise but demanding: the user has to understand the engine before they can express the idea. Generative interfaces invert that relationship. They can begin with the user's rough model, then expose the structure needed to make it testable.
The difficult part is not producing a visually plausible scene. It is maintaining a reliable contract between an ambiguous sketch, the generated scene description, and the simulator that must execute it. That boundary, where language becomes dynamics, is the part of the project I found most interesting.