Projekt INF

INF | Collaboration Infrastructure

Prof. Thomas Ertl, University of Stuttgart
Email | Website

Prof. Falk Schreiber, University of Konstanz
Email | Website

There is no advisor for this project.

Florian Frieß, University of Stuttgart – Email | Website

Dimitar Garkov, University of Konstanz – Email | Website

About this project
Results

In the first funding period, Project INF supported the other projects of the SFB-TRR 161 by providing a central approach to data management and an infrastructure for virtual meetings in a large, high-resolution display scenario.
The project will continue to do so in the upcoming funding period by employing the existing hardware and the software infrastructure which was developed in the project. It will also continue to manage accounts and access rights as necessary. Project INF provides the other projects with guidelines on how to use the research data management system. Since there are many projects in the SFB-TRR 161 which conduct user studies—and therefore handle personal data—Project INF will make sure, that all projects provide a GDPR-compliant documentation of processing activities and processes for protecting personal data.

A dedicated tele-conferencing solution developed during the last funding period was, and is being regularly used in the lecture series and other invited talks. To include researchers at other sites (e.g. Ulm and Munich), who are not able to join the current solution, an RTSP server will be added to the tele-conferencing solution. With this researchers will be able to follow the lecture series and other talks using any standard media player.

An increasing number of research groups in the SFB-TRR 161 are conducting research related to the overall topic of immersion, including a broad range of aspects such as visualization, rendering, interaction, and algorithmic implications. In order to support collaboration, Project INF plans to develop and deploy a new infrastructure that allows research to be conducted not only for, but also in virtual-reality (VR), and augmented-reality (AR) environments, including intermediate forms of mixed-reality (MR). This will strongly improve the opportunities for collaborative research between projects of the SFB-TRR 161, and at the same time foster immersive analytics research, remote collaborative work, and data analysis in immersive environments. The development and deployment of such shared infrastructure will reduce the need for coordination and travel, allow other projects to set up similar and comparable immersive environments without much effort, ease issues of equipment access, as well as directly influence designs and studies for collaborative immersive analytics research. In addition to these direct impacts, research results can then be more easily presented, and also various impressive outreach activities are conceivable. To support natural collaborative interaction, depending on the type of environment, technical solutions that track people, gestures, and devices, and allow us to record and stream voice and camera data will be applied.

Overview of the collaborative infrastructures and their network.

Fig. 1: Talk at the Powerwall at the University of Konstanz

The RTSP streaming solution for the tele-conferencing solution has been implemented and is now regularly used for the lecture series and other invited talks.

Publications

V. Bruder, C. Müller, S. Frey, and T. Ertl, “On Evaluating Runtime Performance of Interactive Visualizations,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, pp. 2848–2862, Sep. 2020, doi: 10.1109/TVCG.2019.2898435.
Abstract
As our field matures, evaluation of visualization techniques has extended from reporting runtime performance to studying user behavior. Consequently, many methodologies and best practices for user studies have evolved. While maintaining interactivity continues to be crucial for the exploration of large data sets, no similar methodological foundation for evaluating runtime performance has been developed. Our analysis of 50 recent visualization papers on new or improved techniques for rendering volumes or particles indicates that only a very limited set of parameters like different data sets, camera paths, viewport sizes, and GPUs are investigated, which make comparison with other techniques or generalization to other parameter ranges at least questionable. To derive a deeper understanding of qualitative runtime behavior and quantitative parameter dependencies, we developed a framework for the most exhaustive performance evaluation of volume and particle visualization techniques that we are aware of, including millions of measurements on ten different GPUs. This paper reports on our insights from statistical analysis of this data discussing independent and linear parameter behavior and non-obvious effects. We give recommendations for best practices when evaluating runtime performance of scientific visualization applications, which can serve as a starting point for more elaborate models of performance quantification.
BibTeX
@article{vtoappearevaluating, abstract = {As our field matures, evaluation of visualization techniques has extended from reporting runtime performance to studying user behavior. Consequently, many methodologies and best practices for user studies have evolved. While maintaining interactivity continues to be crucial for the exploration of large data sets, no similar methodological foundation for evaluating runtime performance has been developed. Our analysis of 50 recent visualization papers on new or improved techniques for rendering volumes or particles indicates that only a very limited set of parameters like different data sets, camera paths, viewport sizes, and GPUs are investigated, which make comparison with other techniques or generalization to other parameter ranges at least questionable. To derive a deeper understanding of qualitative runtime behavior and quantitative parameter dependencies, we developed a framework for the most exhaustive performance evaluation of volume and particle visualization techniques that we are aware of, including millions of measurements on ten different GPUs. This paper reports on our insights from statistical analysis of this data discussing independent and linear parameter behavior and non-obvious effects. We give recommendations for best practices when evaluating runtime performance of scientific visualization applications, which can serve as a starting point for more elaborate models of performance quantification.}, author = {Bruder, Valentin and Müller, Christoph and Frey, Steffen and Ertl, Thomas}, doi = {10.1109/TVCG.2019.2898435}, journal = {IEEE Transactions on Visualization and Computer Graphics}, month = {9}, pages = {2848-2862}, title = {On Evaluating Runtime Performance of Interactive Visualizations}, url = {https://ieeexplore.ieee.org/document/8637795}, volume = 26, year = 2020 }
Link
https://ieeexplore.ieee.org/document/8637795
F. Frieß, M. Braun, V. Bruder, S. Frey, G. Reina, and T. Ertl, “Foveated Encoding for Large High-Resolution Displays,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, Art. no. 2, 2020, doi: 10.1109/TVCG.2020.3030445.
Abstract
Collaborative exploration of scientific data sets across large high-resolution displays requires both high visual detail as wellas low-latency transfer of image data (oftentimes inducing the need to trade one for the other). In this work, we present a system thatdynamically adapts the encoding quality in such systems in a way that reduces the required bandwidth without impacting the detailsperceived by one or more observers. Humans perceive sharp, colourful details, in the small foveal region around the centre of the fieldof view, while information in the periphery is perceived blurred and colourless. We account for this by tracking the gaze of observers,and respectively adapting the quality parameter of each macroblock used by the H.264 encoder, considering the so-called visual acuityfall-off. This allows to substantially reduce the required bandwidth with barely noticeable changes in visual quality, which is crucial forcollaborative analysis across display walls at different locations. We demonstrate the reduced overall required bandwidth and the highquality inside the foveated regions using particle rendering and parallel coordinate
BibTeX
@article{9224182, abstract = {Collaborative exploration of scientific data sets across large high-resolution displays requires both high visual detail as wellas low-latency transfer of image data (oftentimes inducing the need to trade one for the other). In this work, we present a system thatdynamically adapts the encoding quality in such systems in a way that reduces the required bandwidth without impacting the detailsperceived by one or more observers. Humans perceive sharp, colourful details, in the small foveal region around the centre of the fieldof view, while information in the periphery is perceived blurred and colourless. We account for this by tracking the gaze of observers,and respectively adapting the quality parameter of each macroblock used by the H.264 encoder, considering the so-called visual acuityfall-off. This allows to substantially reduce the required bandwidth with barely noticeable changes in visual quality, which is crucial forcollaborative analysis across display walls at different locations. We demonstrate the reduced overall required bandwidth and the highquality inside the foveated regions using particle rendering and parallel coordinate}, author = {Frieß, F. and Braun, M. and Bruder, V. and Frey, S. and Reina, G. and Ertl, T.}, doi = {10.1109/TVCG.2020.3030445}, journal = {IEEE Transactions on Visualization and Computer Graphics}, number = 2, pages = {1-10}, title = {Foveated Encoding for Large High-Resolution Displays}, url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9224182}, volume = 27, year = 2020 }
Link
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9224182
F. Frieß, C. Müller, and T. Ertl, “Real-Time High-Resolution Visualisation,” in Proceedings of the Eurographics Symposium on Vision, Modeling, and Visualization (VMV), 2020, pp. 127–135. doi: 10.2312/vmv.20201195.
Abstract
While visualisation often strives for abstraction, the interactive exploration of large scientific data sets like densely sampled 3Dfields or massive particle data sets still benefits from rendering their graphical representation in large detail on high-resolutiondisplays such as Powerwalls or tiled display walls driven by multiple GPUs or even GPU clusters. Such visualisation systemsare typically rather unique in their setup of hardware and software which makes transferring a visualisation application fromone high-resolution system to another one a complicated task. As more and more such visualisation systems get installed,collaboration becomes desirable in the sense of sharing such a visualisation running on one site in real time with another high-resolution display on a remote site while at the same time communicating via video and audio. Since typical video conferencesolutions or web-based collaboration tools often cannot deal with resolutions exceeding 4K, with stereo displays or with multi-GPU setups, we designed and implemented a new system based on state-of-the-art hardware and software technologies totransmit high-resolution visualisations including video and audio streams via the internet to remote large displays and back.Our system architecture is built on efficient capturing, encoding and transmission of pixel streams and thus supports a multitudeof configurations combining audio and video streams in a generic approac
BibTeX
@inproceedings{v.20201195, abstract = {While visualisation often strives for abstraction, the interactive exploration of large scientific data sets like densely sampled 3Dfields or massive particle data sets still benefits from rendering their graphical representation in large detail on high-resolutiondisplays such as Powerwalls or tiled display walls driven by multiple GPUs or even GPU clusters. Such visualisation systemsare typically rather unique in their setup of hardware and software which makes transferring a visualisation application fromone high-resolution system to another one a complicated task. As more and more such visualisation systems get installed,collaboration becomes desirable in the sense of sharing such a visualisation running on one site in real time with another high-resolution display on a remote site while at the same time communicating via video and audio. Since typical video conferencesolutions or web-based collaboration tools often cannot deal with resolutions exceeding 4K, with stereo displays or with multi-GPU setups, we designed and implemented a new system based on state-of-the-art hardware and software technologies totransmit high-resolution visualisations including video and audio streams via the internet to remote large displays and back.Our system architecture is built on efficient capturing, encoding and transmission of pixel streams and thus supports a multitudeof configurations combining audio and video streams in a generic approac}, author = {Frieß, Florian and Müller, Christoph and Ertl, Thomas}, booktitle = {Proceedings of the Eurographics Symposium on Vision, Modeling, and Visualization (VMV)}, doi = {10.2312/vmv.20201195}, editor = {Krüger, Jens and Niessner, Matthias and Stückler, Jörg}, isbn = {978-3-03868-123-6}, pages = {127-135}, publisher = {The Eurographics Association}, title = {Real-Time High-Resolution Visualisation}, url = {https://doi.org/10.2312/vmv.20201195 }, year = 2020 }
Link
https://doi.org/10.2312/vmv.20201195
C. Müller, M. Braun, and T. Ertl, “Optimised Molecular Graphics on the HoloLens,” in IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019, Osaka, Japan, March 23-27, 2019, 2019, pp. 97–102. doi: 10.1109/VR.2019.8798111.
Abstract
The advent of modern and affordable augmented reality head sets like Microsoft HoloLens has sparked new interest in using virtual and augmented reality technology in the analysis of molecular data. For all visualisation in immersive, mixed-reality scenarios, a sufficiently high rendering speed is an important factor, which leads to the issue of limited processing power available on fully untethered devices facing the situation of handling computationally expensive visualisations. Recent research shows that the space-filling model of even small data sets from the Protein Data Bank (PDB) cannot be rendered at desirable frame rates on the HoloLens. In this work, we report on how to improve the rendering speed of atom-based visualisation of proteins and how the rendering of more abstract representations of the molecules compares against it. We complement our findings with in-depth GPU and CPU performance numbers.
BibTeX
@inproceedings{DBLP:conf/vr/MuilerBE19, abstract = {The advent of modern and affordable augmented reality head sets like Microsoft HoloLens has sparked new interest in using virtual and augmented reality technology in the analysis of molecular data. For all visualisation in immersive, mixed-reality scenarios, a sufficiently high rendering speed is an important factor, which leads to the issue of limited processing power available on fully untethered devices facing the situation of handling computationally expensive visualisations. Recent research shows that the space-filling model of even small data sets from the Protein Data Bank (PDB) cannot be rendered at desirable frame rates on the HoloLens. In this work, we report on how to improve the rendering speed of atom-based visualisation of proteins and how the rendering of more abstract representations of the molecules compares against it. We complement our findings with in-depth GPU and CPU performance numbers.}, author = {Müller, Christoph and Braun, Matthias and Ertl, Thomas}, bibsource = {dblp computer science bibliography, https://dblp.org}, booktitle = {IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019, Osaka, Japan, March 23-27, 2019}, doi = {10.1109/VR.2019.8798111}, pages = {97–102}, publisher = {IEEE}, title = {Optimised Molecular Graphics on the HoloLens}, url = {https://doi.org/10.1109/VR.2019.8798111}, year = 2019 }
Link
https://doi.org/10.1109/VR.2019.8798111
F. Frieß, M. Landwehr, V. Bruder, S. Frey, and T. Ertl, “Adaptive Encoder Settings for Interactive Remote Visualisation on High-Resolution Displays,” in Proceedings of the IEEE Symposium on Large Data Analysis and Visualization - Short Papers (LDAV), 2018, pp. 87–91. doi: 10.1109/LDAV.2018.8739215.
Abstract
We present an approach that dynamically adapts encoder settings for image tiles to yield the best possible quality for a given bandwidth. This reduces the overall size of the image while preserving details. Our application determines the encoding settings in two steps. In the first step, we predict the quality and size of the tiles for different encoding settings using a convolutional neural network. In the second step, we assign the optimal encoder setting to each tile, so that the overall size of the image is lower than a predetermined threshold. Commonly, for tiles that contain complicated structures, a high quality setting is used in order to prevent major information loss, while quality settings are lowered for others to keep the size below the threshold. We demonstrate that we can reduce the overall size of the image while preserving the details in areas of interest using the example of both particle and volume visualisation applications.
BibTeX
@inproceedings{conf/ldav/FriessLBFE18, abstract = {We present an approach that dynamically adapts encoder settings for image tiles to yield the best possible quality for a given bandwidth. This reduces the overall size of the image while preserving details. Our application determines the encoding settings in two steps. In the first step, we predict the quality and size of the tiles for different encoding settings using a convolutional neural network. In the second step, we assign the optimal encoder setting to each tile, so that the overall size of the image is lower than a predetermined threshold. Commonly, for tiles that contain complicated structures, a high quality setting is used in order to prevent major information loss, while quality settings are lowered for others to keep the size below the threshold. We demonstrate that we can reduce the overall size of the image while preserving the details in areas of interest using the example of both particle and volume visualisation applications.}, author = {Frieß, Florian and Landwehr, Mathias and Bruder, Valentin and Frey, Steffen and Ertl, Thomas}, booktitle = {Proceedings of the IEEE Symposium on Large Data Analysis and Visualization - Short Papers (LDAV)}, doi = {10.1109/LDAV.2018.8739215}, ee = {https://doi.org/10.1109/LDAV.2018.8739215}, isbn = {978-1-5386-6873-3}, pages = {87-91}, publisher = {IEEE}, title = {Adaptive Encoder Settings for Interactive Remote Visualisation on High-Resolution Displays}, url = {https://ieeexplore.ieee.org/document/8739215}, year = 2018 }
Link
https://ieeexplore.ieee.org/document/8739215