A02 | Quantifying Visual Computing Systems

Prof. Thomas Ertl, University of Stuttgart
Email | Website

Steffen Frey, University of Groningen (until 06/2020, then external collaborator)
Email | Website

Thomas Ertl
New image

Prof. Melanie Herschel, University of Stuttgart
Email | Website

Melanie Herschel

Moritz Heinemann, University of Stuttgart – Email  Website

The long-term goal of this project was to quantify visual computing systems, i.e., to assess, model, and ultimatelypredict important characteristics that have a substantial impact on user experience. We set out to use these models for algorithmic optimizations and cost savings, e.g., to maintain interactive frame rates during visual exploration of large data sets. While the focus during the first funding period was on runtime performance, this was extended to other performance metrics including throughput, perceived image quality, and energy consumption in the second period. In particular, the trade-offs between different metrics were a key point of investigation. Another major goal was comparing different visual computing methods in this regard. We identified two key visual computing scenarios that should be used as applications scenarios due to their current and projected relevance in the future: scientific visualization in high-performance environments and virtual and augmented reality.

Research Questions

How can we extend performance models from the computer architecture to deal with heterogeneous visual computing architectures and interactive applications?

How can we find the adequate level of abstraction and the relevant parameters for making quantitative performance predictions?

Can our framework support an application to adapt to variable loads and conditions in real time?

Can we extend the model to the adaptive algorithms and the perceptual metrics investigated in other research projects?

How can our models be used to give guarantees for minimal frame rates or maximal interaction latencies for specific data sets?

How can uncertainty be dealt with in terms of both the measurements and the predicted outcome?

Fig. 1: Illustration of our foveated encoding approach for large high-resolution displays.

Fig. 2: Experiment comparing inline, in transit and our hybrid configuration using a Cloverleaf3D simulation and Cinema database generation.

Fig. 3: The test bench used in our power measurement experiments: All significant power rails of the system are redirected through microcontroller sensors. Additionally, an external power analyzer and oscilloscopes can be used for additional measurements and verification.

Publications

  1. C. Müller, M. Heinemann, D. Weiskopf, and T. Ertl, “Power Overwhelming: Quantifying the Energy Cost of Visualisation,” in Proceedings of the 2022 IEEE Workshop on Evaluation and Beyond - Methodological Approaches for Visualization (BELIV), in Proceedings of the 2022 IEEE Workshop on Evaluation and Beyond - Methodological Approaches for Visualization (BELIV). Oct. 2022, pp. 38–46. doi: 10.1109/BELIV57783.2022.00009.
  2. V. Bruder, M. Larsen, T. Ertl, H. Childs, and S. Frey, “A Hybrid In Situ Approach for Cost Efficient Image Database Generation,” IEEE Transactions on Visualization and Computer Graphics, pp. 1–1, 2022, doi: 10.1109/TVCG.2022.3169590.
  3. S. Frey et al., “Parameter Adaptation In Situ: Design Impacts and Trade-Offs,” in In Situ Visualization for Computational Science, H. Childs, J. C. Bennett, and C. Garth, Eds., in In Situ Visualization for Computational Science. Cham: Springer International Publishing, 2022, pp. 159--182. doi: 10.1007/978-3-030-81627-8_8.
  4. H. Tarner, V. Bruder, T. Ertl, S. Frey, and F. Beck, “Visually Comparing Rendering Performance from Multiple Perspectives,” in Vision, Modeling, and Visualization, J. Bender, M. Botsch, and D. Keim, Eds., in Vision, Modeling, and Visualization. The Eurographics Association, 2022. doi: 10.2312/vmv.20221211.
  5. K. Schatz et al., “2019 IEEE Scientific Visualization Contest Winner: Visual Analysis of Structure Formation in Cosmic Evolution,” IEEE Computer Graphics and Applications, vol. 41, no. 6, Art. no. 6, 2021, doi: 10.1109/MCG.2020.3004613.
  6. F. Frieß, M. Braun, V. Bruder, S. Frey, G. Reina, and T. Ertl, “Foveated Encoding for Large High-Resolution Displays,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, Art. no. 2, 2021, doi: 10.1109/TVCG.2020.3030445.
  7. V. Bruder, C. Müller, S. Frey, and T. Ertl, “On Evaluating Runtime Performance of Interactive Visualizations,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, pp. 2848–2862, Sep. 2020, doi: 10.1109/TVCG.2019.2898435.
  8. V. Bruder, K. Kurzhals, S. Frey, D. Weiskopf, and T. Ertl, “Space-Time Volume Visualization of Gaze and Stimulus,” in Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA), K. Krejtz and B. Sharif, Eds., in Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA). ACM, 2019, pp. 12:1-12:9. doi: 10.1145/3314111.3319812.
  9. K. Schatz et al., “Visual Analysis of Structure Formation in Cosmic Evolution,” in Proceedings of the IEEE Scientific Visualization Conference (SciVis), in Proceedings of the IEEE Scientific Visualization Conference (SciVis). 2019, pp. 33–41. doi: 10.1109/scivis47405.2019.8968855.
  10. V. Bruder, C. Schulz, R. Bauer, S. Frey, D. Weiskopf, and T. Ertl, “Voronoi-Based Foveated Volume Rendering,” in Proceedings of the Eurographics Conference on Visualization - Short Papers (EuroVis), J. Johansson, F. Sadlo, and G. E. Marai, Eds., in Proceedings of the Eurographics Conference on Visualization - Short Papers (EuroVis). Eurographics Association, 2019, pp. 67–71. doi: 10.2312/evs.20191172.
  11. H. Zhang, S. Frey, H. Steeb, D. Uribe, T. Ertl, and W. Wang, “Visualization of Bubble Formation in Porous Media,” IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 1, Art. no. 1, 2019, doi: 10.1109/TVCG.2018.2864506.
  12. V. Bruder et al., “Volume-Based Large Dynamic Graph Analysis Supported by Evolution Provenance,” Multimedia Tools and Applications, vol. 78, no. 23, Art. no. 23, 2019, doi: 10.1007/s11042-019-07878-6.
  13. C. Müller, M. Braun, and T. Ertl, “Optimised Molecular Graphics on the HoloLens,” in IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019, Osaka, Japan, March 23-27, 2019, in IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019, Osaka, Japan, March 23-27, 2019. IEEE, 2019, pp. 97–102. doi: 10.1109/VR.2019.8798111.
  14. V. Bruder, M. Hlawatsch, S. Frey, M. Burch, D. Weiskopf, and T. Ertl, “Volume-Based Large Dynamic Graph Analytics,” in Proceedings of the International Conference Information Visualisation (IV), E. Banissi, R. Francese, M. W. McK. Bannatyne, T. G. Wyeld, M. Sarfraz, J. M. Pires, A. Ursyn, F. Bouali, N. Datia, G. Venturini, G. Polese, V. Deufemia, T. D. Mascio, M. Temperini, F. Sciarrone, D. Malandrino, R. Zaccagnino, P. Díaz, F. Papadopoulo, A. F. Anta, A. Cuzzocrea, M. Risi, U. Erra, and V. Rossano, Eds., in Proceedings of the International Conference Information Visualisation (IV). IEEE, 2018, pp. 210–219. doi: 10.1109/iV.2018.00045.
  15. F. Frieß, M. Landwehr, V. Bruder, S. Frey, and T. Ertl, “Adaptive Encoder Settings for Interactive Remote Visualisation on High-Resolution Displays,” in Proceedings of the IEEE Symposium on Large Data Analysis and Visualization - Short Papers (LDAV), in Proceedings of the IEEE Symposium on Large Data Analysis and Visualization - Short Papers (LDAV). IEEE, 2018, pp. 87–91. doi: 10.1109/LDAV.2018.8739215.
  16. S. Frey, “Spatio-Temporal Contours from Deep Volume Raycasting,” Computer Graphics Forum, vol. 37, no. 3, Art. no. 3, 2018, doi: 10.1111/cgf.13438.
  17. S. Frey, “Sampling and Estimation of Pairwise Similarity in Spatio-Temporal Data Based on Neural Networks,” in Informatics, in Informatics, vol. 4. Multidisciplinary Digital Publishing Institute (MDPI), 2017, p. 27. doi: 10.3390/informatics4030027.
  18. M. Heinemann, V. Bruder, S. Frey, and T. Ertl, “Power Efficiency of Volume Raycasting on Mobile Devices,” in Proceedings of the Eurographics Conference on Visualization (EuroVis) - Poster Track, in Proceedings of the Eurographics Conference on Visualization (EuroVis) - Poster Track. 2017. doi: 10.2312/eurp.20171166.
  19. S. Frey and T. Ertl, “Flow-Based Temporal Selection for Interactive Volume Visualization,” Computer Graphics Forum, vol. 36, no. 8, Art. no. 8, 2017, doi: 10.1111/cgf.13070.
  20. S. Frey and T. Ertl, “Progressive Direct Volume-to-Volume Transformation,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 1, Art. no. 1, 2017, doi: 10.1109/TVCG.2016.2599042.
  21. G. Tkachev, S. Frey, C. Müller, V. Bruder, and T. Ertl, “Prediction of Distributed Volume Visualization Performance to Support Render Hardware Acquisition,” in Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), in Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization (EGPGV). Eurographics Association, 2017, pp. 11–20. doi: 10.2312/pgv.20171089.
  22. V. Bruder, S. Frey, and T. Ertl, “Prediction-Based Load Balancing and Resolution Tuning for Interactive Volume Raycasting,” Visual Informatics, vol. 1, no. 2, Art. no. 2, 2017, doi: 10.1016/j.visinf.2017.09.001.
  23. V. Bruder, S. Frey, and T. Ertl, “Real-Time Performance Prediction and Tuning for Interactive Volume Raycasting,” in Proceedings of the SIGGRAPH Asia Symposium on Visualization, ACM, Ed., in Proceedings of the SIGGRAPH Asia Symposium on Visualization. ACM, 2016, pp. 1–8. doi: 10.1145/3002151.3002156.
  24. C. Schulz et al., “Generative Data Models for Validation and Evaluation of Visualization Techniques,” in Proceedings of the Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization (BELIV), in Proceedings of the Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization (BELIV). ACM, 2016, pp. 112–124. doi: 10.1145/2993901.2993907.
  25. S. Frey and T. Ertl, “Auto-Tuning Intermediate Representations for In Situ Visualization,” in Proceedings of the New York Scientific Data Summit (NYSDS), in Proceedings of the New York Scientific Data Summit (NYSDS). IEEE, 2016, pp. 1–10. doi: 10.1109/NYSDS.2016.7747807.
  26. S. Frey, F. Sadlo, and T. Ertl, “Balanced Sampling and Compression for Remote Visualization,” in Proceedings of the SIGGRAPH Asia Symposium on High Performance Computing, in Proceedings of the SIGGRAPH Asia Symposium on High Performance Computing. ACM, 2015, pp. 1–4. doi: 10.1145/2818517.2818529.

Project Group A

Models and Measures

 

Completed

 

Project Group B

Adaptive Algorithms

 

Completed

 

Project Group C

Interaction

 

Completed

 

Project Group D

Applications

 

Completed