Research

A world to practise in, from a photograph

A team of agents turns one photograph of a real place into a simulated world a policy can train in, generated entirely in code.

4 August 2026

A warehouse aisle, photographed on a phone — Left, the photograph a contact sent us. Right, the aisle rebuilt in code, from the camera position solved out of that photograph.

The same aisle, rebuilt in code — Left, the photograph a contact sent us. Right, the aisle rebuilt in code, from the camera position solved out of that photograph.

Training a policy in simulation needs environments that match the site the robot will work in, and building those by hand is the slow, expensive step. So we do not build them by hand. A photograph goes in, and a scene a robot can be dropped into and trained in comes out, generated entirely from primitives at load time: 4,262 meshes and 179,708 triangles, no imported models, no scanned geometry. Because it is code, the same aisle re-samples at another stock density, another light, another length.

Written by agents

The scene is written by agents working in rounds. A round produces a specification, builds the scene from it, renders it, scores the render against the photograph, and hands what it learned to the next round. In exhaustive mode several candidate scenes are generated in parallel and scored against one another, and every run is kept and numbered, so a scene's convergence is inspectable pass by pass.

Each pass writes a new specification, builds it, renders it, and scores the render against the photograph, then revises. In exhaustive mode several candidate scenes run in parallel and compete for the next round.

Gates that can fail

The loop on its own did not work. Objects were specified in prose, the agents built exactly what the prose said, and the prose was wrong: a cabinet in the walkway, pallets rotated ninety degrees, a portal frame buried inside a shelving run. An agent reviewing its own render agrees with itself. Nothing improved until every claim had to survive a check that returns pass or fail, and both of the geometry faults these gates were built for were then caught by measurement, before anyone looked at a render.

Collisions cross-assembly interpenetration, from real world-space bounding boxes
Layout overlapping footprints and a blocked walkway
Score agreement with the photograph, signal by signal
Path trace the photoreal mode still builds and renders

Recovering the camera

An aisle is a corridor of mutually parallel lines: rack beams, light runs, conveyor rails, floor joints. Their common vanishing point in the photograph is the camera's orientation. The catch is that the strongest set of parallel lines in this frame is the conveyor's rollers, which run across the aisle and converge far outside it. Fitting to them once points the camera about sixty degrees the wrong way, so the solver peels the pencils of lines apart and keeps the one that converges inside the frame.

The aisle's parallel lines meet at one point, and that point is where the camera was looking. Two independently recovered sets of lines agreed to within half a degree.

Why it matters

Building an environment that matches a customer's site by hand takes days. From one photograph it takes minutes, and that speed is what a 24 hour path from site scan to deployed policy depends on. Because the aisle is code rather than a captured mesh, the same generator also produces the variation training needs: the shelves restocked, the lighting changed, the run made longer, without another visit.

What this is not

This is a reconstruction, not a measurement. Geometry is inferred from the photograph, so dimensions are plausible rather than surveyed, and anything the camera could not see is generated. It is built for training and rehearsal, and should not be used for metrology, clearance checks or safety layout. Procedural geometry has a ceiling: this is strong architectural fidelity, not a pixel match to the photograph.