Summary:
VENI models natural lighting directly on the sphere with rotation-aware architecture and a structured latent space, offering more realistic and controllable illumination for inverse rendering and graphics applications.
Researchers from Friedrich-Alexander University Erlangen-Nürnberg, the University of York, and startup pxld.ai have unveiled VENI, a novel model for capturing natural illumination in computer vision and graphics. VENI addresses one of the field’s most persistent challenges: recovering lighting, materials, and geometry from images, also known as inverse rendering.
Traditional methods struggle because real-world lighting is inherently ambiguous, and many approaches rely on 2D projections or indirect tricks, such as reconstructing panoramic environments from small image crops or using reflective objects to infer lighting. These methods often produce distortions or require partial visibility of the lighting environment.
VENI introduces a clean, physically realistic solution: it models illumination directly on a sphere, avoiding the geometric distortions of flattening a spherical world into 2D. The model leverages a Vector Neuron Transformer (VN-Transformer) architecture, which makes it naturally aware of rotations. This means the system recognizes that turning a lighting environment does not change its physical validity, eliminating the need for complex data augmentation.
Additionally, VENI employs a variational autoencoder (VAE) to create a structured and unique latent space for lighting. This enables smooth interpolation between lighting conditions and predictable control, overcoming limitations of prior models where the same lighting could be represented in multiple ways.
By combining spherical illumination modeling, rotation-aware architecture, and a clean latent space, VENI captures rich, realistic lighting with fewer parameters. The approach lays a strong foundation for inverse rendering, relighting, virtual reality, digital humans, and autonomous perception, bringing computer-generated lighting closer to real-world realism.
