A07 | Visual Attention Modeling for Optimization of Information Visualizations

Prof. Andreas Bulling, University of Stuttgart
Email | Website

Andreas Bulling

Dr. Lewis L. Chuang, LMU Munich
Email | Website

Lewis L. Chuang

Yao Wang, University of Stuttgart – Email | Website 

Despite the importance of human vision for perceiving and understanding information visualizations, dominant approaches to quantify users’ visual attention require special-purpose eye tracking equipment. However, such equipment is not always available, has to be calibrated to each user, and is limited to post-hoc optimization of information visualizations.

This project aims to integrate automatic quantification of spatio-temporal visual attention directly into the visualization design process without the need for any eye tracking equipment. To achieve these goals, the project takes inspiration from computational models of visual attention (saliency models) that mimic basic perceptual processing to reproduce attentive behavior. Originally introduced in computational neuroscience, saliency models have been tremendously successful in several research fields, particularly in computer vision. In contrast, few works have investigated the use of saliency models in information visualization. We will develop new methods for data-driven attention prediction for information visualizations as well as joint modeling of bottom-up and top-down visual attention, and use them to investigate attention-driven optimization and real-time adaptation of static and dynamic information visualizations.

Research Questions

How can we predict attention on a wide range of real-world information visualizations?

Which optimisation goals and mechanisms are required to optimise static information visualisations?

How can we jointly model bottom-up and top-down attention on dynamic information visualisations?

Which mechanisms are required to adapt visual analytics interfaces to users' attention?

Sample Attention Maps

Sample attention maps from different models compared to the ground truth obtained using an eye tracker.

Our maps closely match the ground truth while existing approaches fail to incorporate important user interace areas.


  1. F. Chiossi et al., “Adapting visualizations and interfaces to the user,” it - Information Technology, vol. 64, no. 4–5, Art. no. 4–5, 2022, doi: 10.1515/itit-2022-0035.
  2. Y. Wang, M. Koch, M. Bâce, D. Weiskopf, and A. Bulling, “Impact of Gaze Uncertainty on AOIs in Information Visualisations,” in 2022 Symposium on Eye Tracking Research and Applications, Jun. 2022, pp. 1–6. doi: 10.1145/3517031.3531166.
  3. Y. Wang, C. Jiao, M. Bâce, and A. Bulling, “VisRecall: Quantifying Information Visualisation Recallability Via Question Answering,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 12, Art. no. 12, 2022, doi: 10.1109/TVCG.2022.3198163.
  4. Y. Wang, M. Bâce, and A. Bulling, “Scanpath Prediction on Information Visualisations,” 2022. doi: 10.48550/arXiv.2112.02340.
  5. K. Kurzhals et al., “Visual Analytics and Annotation of Pervasive Eye Tracking Video,” in Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA), Stuttgart, Germany, 2020, pp. 16:1-16:9. doi: 10.1145/3379155.3391326.
  6. H. Sattar, A. Bulling, and M. Fritz, “Predicting the Category and Attributes of Visual Search Targets Using Deep Gaze Pooling,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW), 2017, pp. 2740–2748. doi: 10.1109/ICCVW.2017.322.
  7. X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 1, Art. no. 1, 2017, doi: 10.1109/TPAMI.2017.2778103.
  8. M. Tonsen, J. Steil, Y. Sugano, and A. Bulling, “InvisibleEye: Mobile Eye Tracking Using Multiple Low-Resolution Cameras and Learning-Based Gaze Estimation,” in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2017, vol. 1, no. 3, pp. 106:1-106:21. doi: https://doi.org/10.1145/3130971.
  9. X. Zhang, Y. Sugano, and A. Bulling, “Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery,” in Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2017, pp. 193–203. doi: 10.1145/3126594.3126614.
  10. P. Xu, Y. Sugano, and A. Bulling, “Spatio-Temporal Modeling and Prediction of Visual Attention in Graphical User Interfaces,” in Proceedings of the CHI Conference on Human Factors in Computing Systems, 2016, pp. 3299–3310. doi: 10.1145/2858036.2858479.
  11. E. Wood, T. Baltrusaitis, L.-P. Morency, P. Robinson, and A. Bulling, “Learning an Appearance-Based Gaze Estimator from One Million Synthesised Images,” in Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA), 2016, pp. 131–138. doi: 10.1145/2857491.2857492.
  12. E. Wood, T. Baltrusaitis, L.-P. Morency, P. Robinson, and A. Bulling, “A 3D Morphable Eye Region Model for Gaze Estimation,” in Proceedings of the European Conference on Computer Vision (ECCV), 2016, pp. 297–313. doi: 10.1007/978-3-319-46448-0_18.
  13. X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2299–2308. doi: 10.1109/CVPRW.2017.284.