UniLight: A Unified Representation for Lighting
Lighting has a strong influence on visual appearance, yet understanding and representing lighting in images remains notoriously difficult. Various lighting representations exist, such as environment maps, irradiance, spherical harmonics, or text, but they are incompatible, which limits cross-modal transfer. We thus propose UniLight, a joint latent space as lighting representation, that unifies multiple modalities within a shared embedding. Modality-specific encoders for text, images, irradiance, and environment maps are trained contrastively to align their representations, with an auxiliary spherical-harmonics prediction task reinforcing directional understanding. Our multi-modal data pipeline enables large-scale training and evaluation across three tasks: lighting-based retrieval, environment-map generation, and lighting control in diffusion-based image synthesis. Experiments show that our representation captures consistent and transferable lighting features, enabling flexible manipulation across modalities.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Multi-Agent Reinforcement Learning for Traffic Signal Control through Universal Communication Method
How to coordinate the communication among intersections effectively in real complex traffic scenarios with multi-intersection is challenging. Existing approaches only enable the communication in a heuristic manner withou…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Traffic Signal ControlTUNA: Taming Unified Visual Representations for Native Unified Multimodal Models
Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present TUNA, a native UMM that builds a unified continuous visual representation by cascading…
Video GenerationImage EditingStyleLight: HDR Panorama Generation for Lighting Estimation and Editing
We present a new lighting estimation and editing framework to generate high-dynamic-range (HDR) indoor panorama lighting from a single limited field-of-view (LFOV) image captured by low-dynamic-range (LDR) cameras. Exist…
Lighting EstimationUniVoxel: Fast Inverse Rendering by Unified Voxelization of Scene Representation
Typical inverse rendering methods focus on learning implicit neural scene representations by modeling the geometry, materials and illumination separately, which entails significant computations for optimization. In this …
Inverse RenderingLearning Indoor Inverse Rendering with 3D Spatially-Varying Lighting
In this work, we address the problem of jointly estimating albedo, normals, depth and 3D spatially-varying lighting from a single image. Most existing methods formulate the task as image-to-image translation, ignoring th…
Image-to-Image TranslationInverse RenderingTranslation