Project A06

A06 | Quantitative Quality Assessment of Computational Photography

Prof. Martin Fuchs, Hochschule der Medien
Email | Website

Prof. Bastian Goldlücke, University of Konstanz
Email | Website

Alexander Wender, University of Stuttgart

About this project
Results

Computational photography (CP) has in recent years emerged as a vibrant sub-field within visual computing and has brought forth highly creative techniques, expanding the notion of a camera beyond the traditional model. By combining novel optical designs with algorithmic techniques for image reconstruction, CP techniques provide advantages such as higher image quality (like removal of motion blur) or even post-acquisition control of photographic parameters (e.g., refocusing).

Due to the great variation of imaging approaches, though, quantitative means, which enable direct evaluations of the comparative performance, are largely unavailable to date. In this project, we intend to address this by establishing suitable benchmarks.

Reseach Questions

What scenes can serve as litmus tests for CP techniques?

How can we make them digitally available to the community in efficient ways?

How can we derive predictive performance metrics?

Refocused images, recorded with a household computational camera.
(Alexander Wender, Julian Iseringhausen, Bastian Goldluecke, Martin Fuchs and Matthias B. Hullin. (2015): Light Field Imaging through Household Optics. Vision, Modeling & Visualization).

Publications

J. Iseringhausen et al., “4D Imaging through Spray-on Optics,” ACM Transactions on Graphics, vol. 36, no. 4, Art. no. 4, 2017, doi: 10.1145/3072959.3073589.
Abstract
Light fields are a powerful concept in computational imaging and a mainstay in image-based rendering; however, so far their acquisition required either carefully designed and calibrated optical systems (micro-lens arrays), or multi-camera/multi-shot settings. Here, we show that fully calibrated light field data can be obtained from a single ordinary photograph taken through a partially wetted window. Each drop of water produces a distorted view on the scene, and the challenge of recovering the unknown mapping from pixel coordinates to refracted rays in space is a severely underconstrained problem. The key idea behind our solution is to combine ray tracing and low-level image analysis techniques (extraction of 2D drop contours and locations of scene features seen through drops) with state-of-the-art drop shape simulation and an iterative refinement scheme to enforce photo-consistency across features that are seen in multiple views. This novel approach not only recovers a dense pixel-to-ray mapping, but also the refractive geometry through which the scene is observed, to high accuracy. We therefore anticipate that our inherently self-calibrating scheme might also find applications in other fields, for instance in materials science where the wetting properties of liquids on surfaces are investigated.
BibTeX
@article{iseringhausen2017imaging, abstract = {Light fields are a powerful concept in computational imaging and a mainstay in image-based rendering; however, so far their acquisition required either carefully designed and calibrated optical systems (micro-lens arrays), or multi-camera/multi-shot settings. Here, we show that fully calibrated light field data can be obtained from a single ordinary photograph taken through a partially wetted window. Each drop of water produces a distorted view on the scene, and the challenge of recovering the unknown mapping from pixel coordinates to refracted rays in space is a severely underconstrained problem. The key idea behind our solution is to combine ray tracing and low-level image analysis techniques (extraction of 2D drop contours and locations of scene features seen through drops) with state-of-the-art drop shape simulation and an iterative refinement scheme to enforce photo-consistency across features that are seen in multiple views. This novel approach not only recovers a dense pixel-to-ray mapping, but also the refractive geometry through which the scene is observed, to high accuracy. We therefore anticipate that our inherently self-calibrating scheme might also find applications in other fields, for instance in materials science where the wetting properties of liquids on surfaces are investigated. }, author = {Iseringhausen, Julian and Goldlücke, Bastian and Pesheva, Nina and Iliev, Stanimir and Wender, Alexander and Fuchs, Martin and B. Hullin, Matthias}, booktitle = {ACM Transactions on Graphics}, doi = {10.1145/3072959.3073589}, number = 4, pages = {35:1--35:11}, title = {4D Imaging through Spray-on Optics }, url = {https://doi.org/10.1145/3072959.3073589}, volume = 36, year = 2017 }
Link
https://doi.org/10.1145/3072959.3073589