Real-Time Light Field Reconstruction and Neural Rendering
Building on the foundations of spherical light field reconstruction and rendering, this work introduces a series of progressively more capable and efficient view synthesis networks. These methods evolve from full light field reconstruction to a direct, position-aware approach capable of rendering novel views within a large volume from a single input image at interactive frame rates. To enhance these synthesized views for augmented reality (AR) applications, a unified multimodal network is developed that jointly predicts a new view and its corresponding depth map from a shared latent space. This multimodal approach ensures multi-view and geometric consistency between appearance and geometry, which is crucial for preventing visual artifacts like flickering.
Articles
Real-time position-aware view synthesis from single-view input
Manu Gond, Emin Zerman, Sebastian Knorr, Mårten Sjöström
Signal Processing: Image Communication, 2026
https://doi.org/10.1016/j.image.2026.117699
PVSDNet: Joint Depth Prediction and View Synthesis Via Shared Latent Spaces in Real-Time
Manu Gond, Emin Zerman, Sebastian Knorr, Mårten Sjöström
IEEE Access, 2026.
https://ieeexplore.ieee.org/document/11348070
Real-Time View Synthesis with Multiplane Image Network using Multimodal Supervision
Manu Gond, Mohammadreza Shamshirgarha, Emin Zerman, Sebastian Knorr, Maarten Sjöström
In the proceedings of IEEE 27th International Workshop on Multimedia Signal Processing (MMSP), 2025.
https://doi.org/10.1109/MMSP64401.2025.11324036
A Visual Quality of Experience Toolkit for Realistic Immersive Telepresence Applications
Manu Gond, Emin Zerman, Mohammadreza Shamshirgarha, Sebastian Knorr, Maarten Sjöström
In the proceedings of 17th International Conference on Quality of Multimedia Experience (QoMEX), 2025.
https://doi.org/10.1109/QoMEX65720.2025.11219996
LFSphereNet: Real Time Spherical Light Field Reconstruction from a Single Omnidirectional Image
Manu Gond, Emin Zerman, Sebastian Knorr, Maarten Sjöström
In the proceedings of ACM SIGGRAPH European Conference on Visual Media Production, 2023.
https://dl.acm.org/doi/10.1145/3626495.3626500