Project A04

A04 | Quantitative Models for Visual Abstraction

Prof. Oliver Deussen, University of Konstanz
Email | Website

Prof. Marc Ernst, Ulm University
Email | Website

Dr. Kwan Kin Chung, University of Konstanz

About this project
Results

We aim at finding abstraction methods for visual computing that create graphical representations for given data with a quantitatively determined degree of abstraction. Appropriate abstraction styles will be selected and representations will be developed that allow us to technically quantify the visual representation (e.g. the number of graphical elements used for a representation). In a second step, we will develop methods that perform abstraction also in a perceptually linear way.

This project will provide visual abstraction methods for other projects of the Collaborative Research Center.

Research questions

Can we measure the degree of abstraction of a non-photorealistic rendering?

Can we create abstraction methods with coherence between created abstract representations?

Is it possible to parametrize different abstraction styles in a technically linear way?

Is the parametrization also adjustable in a perceptually linear way?

Representation of an input image (upper left) by illustrations with varying number of graphical elements (24.000, 12.000 and 6.000 stipple points.

Fig. 1: Representation of an input image (upper left) by illustrations with varying number of graphical elements (24.000, 12.000 and 6.000 stipple points.

Publications

D. Laupheimer, P. Tutzauer, N. Haala, and M. Spicker, “Neural Networks for the Classification of Building Use from Street-view Imagery,” ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. IV–2, pp. 177–184, 2018, doi: 10.5194/isprs-annals-IV-2-177-2018.
Abstract
Within this paper we propose an end-to-end approach for classifying terrestrial images of building facades into five different utility classes (commercial, hybrid, residential, specialUse, underConstruction) by using Convolutional Neural Networks (CNNs). For our examples we use images provided by Google Street View. These images are automatically linked to a coarse city model, including the outlines of the buildings as well as their respective use classes. By these means an extensive dataset is available for training and evaluation of our Deep Learning pipeline. The paper describes the implemented end-to-end approach for classifying street-level images of building facades and discusses our experiments with various CNNs. In addition to the classification results, so-called Class Activation Maps (CAMs) are evaluated. These maps give further insights into decisive facade parts that are learned as features during the training process. Furthermore, they can be used for the generation of abstract presentations which facilitate the comprehension of semantic image content. The abstract representations are a result of the stippling method, an importance-based image rendering.
BibTeX
@article{laupheimer2018neural, abstract = {Within this paper we propose an end-to-end approach for classifying terrestrial images of building facades into five different utility classes (commercial, hybrid, residential, specialUse, underConstruction) by using Convolutional Neural Networks (CNNs). For our examples we use images provided by Google Street View. These images are automatically linked to a coarse city model, including the outlines of the buildings as well as their respective use classes. By these means an extensive dataset is available for training and evaluation of our Deep Learning pipeline. The paper describes the implemented end-to-end approach for classifying street-level images of building facades and discusses our experiments with various CNNs. In addition to the classification results, so-called Class Activation Maps (CAMs) are evaluated. These maps give further insights into decisive facade parts that are learned as features during the training process. Furthermore, they can be used for the generation of abstract presentations which facilitate the comprehension of semantic image content. The abstract representations are a result of the stippling method, an importance-based image rendering.}, author = {Laupheimer, D. and Tutzauer, P. and Haala, N. and Spicker, M.}, doi = {10.5194/isprs-annals-IV-2-177-2018}, journal = {ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences }, pages = {177-184}, title = {Neural Networks for the Classification of Building Use from Street-view Imagery }, url = {https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2/177/2018/isprs-annals-IV-2-177-2018.pdf}, volume = {IV-2}, year = 2018 }
Link
https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2/177/2018/isprs-annals-IV-2-177-2018.pdf
M. Spicker, F. Hahn, T. Lindemeier, D. Saupe, and O. Deussen, “Quantifying Visual Abstraction Quality for Stipple Drawings,” in Proceedings of the Symposium on Non-Photorealistic Animation and Rendering (NPAR), 2017, pp. 8:1-8:10, [Online]. Available: https://doi.org/http://dx.doi.org/10.1145/3092919.3092923.
Abstract
We investigate how the perceived abstraction quality of stipple illustrations is related to the number of points used to create them. Since it is difficult to find objective functions that quantify the visual quality of such illustrations, we gather comparative data by a crowdsourcing user study and employ a paired comparison model to deduce absolute quality values. Based on this study we show that it is possible to predict the perceived quality of stippled representations based on the properties of an input image. Our results are related to Weber-Fechner's law from psychophysics and indicate a logarithmic relation between numbers of points and perceived abstraction quality. We give guidance for the number of stipple points that is typically enough to represent an input image well.
BibTeX
@inproceedings{spickerqantifying, abstract = {We investigate how the perceived abstraction quality of stipple illustrations is related to the number of points used to create them. Since it is difficult to find objective functions that quantify the visual quality of such illustrations, we gather comparative data by a crowdsourcing user study and employ a paired comparison model to deduce absolute quality values. Based on this study we show that it is possible to predict the perceived quality of stippled representations based on the properties of an input image. Our results are related to Weber-Fechner's law from psychophysics and indicate a logarithmic relation between numbers of points and perceived abstraction quality. We give guidance for the number of stipple points that is typically enough to represent an input image well.}, author = {Spicker, Marc and Hahn, Franz and Lindemeier, Thomas and Saupe, Dietmar and Deussen, Oliver}, booktitle = {Proceedings of the Symposium on Non-Photorealistic Animation and Rendering (NPAR)}, editor = {ACM}, pages = {8:1-8:10}, publisher = {Association for Computing Machinery}, title = {Quantifying Visual Abstraction Quality for Stipple Drawings}, url = {https://doi.org/http://dx.doi.org/10.1145/3092919.3092923}, year = 2017 }
Link
https://doi.org/http://dx.doi.org/10.1145/3092919.3092923
J. Kratt, F. Eisenkeil, M. Spicker, Y. Wang, D. Weiskopf, and O. Deussen, “Structure-aware Stylization of Mountainous Terrains,” in Vision, Modeling & Visualization, M. Hullin, R. Klein, T. Schultz, and A. Yao, Eds. The Eurographics Association, 2017.
- BibTeX
- Link
BibTeX
@inbook{vmv.20171255, author = {Kratt, Julian and Eisenkeil, Ferdinand and Spicker, Marc and Wang, Yunhai and Weiskopf, Daniel and Deussen, Oliver}, booktitle = {Vision, Modeling & Visualization}, doi = {10.2312/vmv.20171255}, editor = {Hullin, Matthias and Klein, Reinhard and Schultz, Thomas and Yao, Angela}, isbn = {978-3-03868-049-9}, publisher = {The Eurographics Association}, title = {Structure-aware Stylization of Mountainous Terrains}, url = {https://diglib.eg.org/handle/10.2312/vmv20171255}, year = 2017 }
Link
https://diglib.eg.org/handle/10.2312/vmv20171255
O. Deussen, M. Spicker, and Q. Zheng, “Weighted Linde-Buzo-Gray Stippling,” ACM Transactions on Graphics, vol. 36, no. 6, Art. no. 6, 2017, doi: 10.1145/3130800.3130819.
- BibTeX
- Link
BibTeX
@article{Deussen2017LindeBuzoGray, author = {Deussen, O. and Spicker, M. and Zheng, Q.}, doi = {10.1145/3130800.3130819}, journal = {ACM Transactions on Graphics}, month = nov, number = 6, pages = {233:1--233:12}, title = {Weighted Linde-Buzo-Gray Stippling}, url = {http://doi.acm.org/10.1145/3130800.3130819}, volume = 36, year = 2017 }
Link
http://doi.acm.org/10.1145/3130800.3130819
P. Tutzauer, S. Becker, T. Niese, O. Deussen, and D. Fritsch, “Understanding Human Perception of Building Categories in Virtual 3d Cities - a User Study,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (ISPRS), vol. XLI-B2, pp. 683–687, 2016, doi: http://dx.doi.org/10.5194/isprs-archives-XLI-B2-683-2016.
Abstract
Virtual 3D cities are becoming increasingly important as a means of visually communicating diverse urban-related information. To get a deeper understanding of a human’s cognitive experience of virtual 3D cities, this paper presents a user study on the human ability to perceive building categories (e.g. residential home, office building, building with shops etc.) from geometric 3D building representations. The study reveals various dependencies between geometric properties of the 3D representations and the perceptibility of the building categories. Knowledge about which geometries are relevant, helpful or obstructive for perceiving a specific building category is derived. The importance and usability of such knowledge is demonstrated based on a perception-guided 3D building abstraction process.
BibTeX
@article{tutzauer2016understanding, abstract = {Virtual 3D cities are becoming increasingly important as a means of visually communicating diverse urban-related information. To get a deeper understanding of a human’s cognitive experience of virtual 3D cities, this paper presents a user study on the human ability to perceive building categories (e.g. residential home, office building, building with shops etc.) from geometric 3D building representations. The study reveals various dependencies between geometric properties of the 3D representations and the perceptibility of the building categories. Knowledge about which geometries are relevant, helpful or obstructive for perceiving a specific building category is derived. The importance and usability of such knowledge is demonstrated based on a perception-guided 3D building abstraction process.}, author = {Tutzauer, Patrick and Becker, Susanne and Niese, Till and Deussen, Oliver and Fritsch, Dieter}, doi = {http://dx.doi.org/10.5194/isprs-archives-XLI-B2-683-2016}, journal = {The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (ISPRS)}, pages = {683-687}, title = {Understanding Human Perception of Building Categories in Virtual 3d Cities - a User Study}, url = {https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLI-B2/683/2016/isprs-archives-XLI-B2-683-2016.pdf}, volume = {XLI-B2}, year = 2016 }
Link
https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLI-B2/683/2016/isprs-archives-XLI-B2-683-2016.pdf
M. Spicker, J. Kratt, D. Arellano, and O. Deussen, “Depth-aware Coherent Line Drawings,” in Proceedings of the SIGGRAPH Asia Symposium on Computer Graphics and Interactive Techniques, Technical Briefs, 2015, pp. 1:1-1:5, [Online]. Available: http://doi.acm.org/10.1145/2820903.2820909.
- BibTeX
- Link
BibTeX
@inproceedings{spicker2015depthaware, author = {Spicker, Marc and Kratt, Julian and Arellano, Diana and Deussen, Oliver}, booktitle = {Proceedings of the SIGGRAPH Asia Symposium on Computer Graphics and Interactive Techniques, Technical Briefs}, pages = {1:1-1:5}, publisher = {ACM}, title = {Depth-aware Coherent Line Drawings}, url = {http://doi.acm.org/10.1145/2820903.2820909}, year = 2015 }
Link
http://doi.acm.org/10.1145/2820903.2820909