as part of the ACM Symposium on Eye Tracking Research & Applications (ETRA), Stuttgart, June 2-5, 2020
Event date:  June 2, 2020 1:00 PM  to June 2, 2020 4:00 PM

Multimedia applications are commonly targeted at the human sensory system. This fact renders subjectively perceived quality the ultimate performance metric in media applications. For visual media such as images and videos, eye tracking provides unprecedented insight into fundamental stages of visual perception. Moreover, it can be expected that eye trackers soon will become standard components in laptops, tablet computers, and smartphones. This allows to deepen our understanding of how humans perceive multimedia, interact with existing multimedia systems, and also to design novel applications that dynamically adapt according to an observer's gaze. In this workshop, we will explore how eye tracking can help to quantify and analyse visual multimedia aspects such as saliency and perceptual quality. ET-MM focuses more on data-driven algorithms and technology, including gaze based interfaces for control of multimedia applications.

 

The workshop will be held online digitally on June 2, 2020 from 13.00h-16.00h CEST (Central European Summer Time). This workshop is part of the ACM Symposium on Eye Tracking Research & Applications (ETRA).

 

Online registration for participation is free and available at eventbrite under this  URL.

 

Program

Welcome / Introduction (5 min)
Lewis Chuang

Session 1 - Keynote (Chair: Lewis Chuang)

Predicting eye movements in images and video: Recent progress and future challenges (40 min)
Thomas Wallis Amazon Tübingen, Tübingen, Germany
For more details see below.

Session 2 - Contributed presentations A (Chair: Hantao Liu)

Sequence Models in Eye Tracking: Predicting Pupil Diameter During Learning (18 min)
Sharath C Koorathota Department of Biomedical Engineering, Columbia University, New York, New York, United States Fractal Media, New York, New York, United States
Kaveri Thakoor Department of Biomedical Engineering, Columbia University, New York, New York, United States
Patrick Adelman Fractal Media, New York, New York, United States
Yaoli Mao Human Development, Columbia University, New York, New York, United States
Xueqing Liu Department of Biomedical Engineering, Columbia University, New York, New York, United States
Paul Sajda Department of Biomedical Engineering, Columbia University, New York, New York, United States

Gaze Estimation in the Dark with Generative Adversarial Networks (18 min)
Jung-Hwa Kim Kumoh National Institute of Technology, Gumi, Korea, Republic of
Jin-Woo Jeong Kumoh National Institute of Technology, Gumi, Korea, Republic of

Analyzing Transferability of Happiness Detection via Gaze Tracking in Multimedia Applications (18 min)
David Bethge Porsche AG, Stuttgart, Germany LMU, Munich, Germany
Lewis L Chuang LMU Munich, Munich, Germany
Tobias Grosse-Puppendahl Porsche AG, Stuttgart, Germany

Break (10 min)

Session 3 - Contributed presentations B (Chair: Dietmar Saupe)

Gaze Data for Quality Assessment of Foveated Video (18 min)
Oliver Wiedemann University of Konstanz, Konstanz, Germany
Dietmar Saupe University of Konstanz, Konstanz, Germany

Toward A Gaze-Enabled Assistance System (18 min)
Kenan Bektas University of St. Gallen, Institute of Computer Science (ICS-HSG), St. Gallen, Switzerland

Implications of Eye Tracking Research to Cinematic Virtual Reality (18 min)
Sylvia Rothe LMU Munich University, Munich, Germany
Lewis L Chuang LMU Munich, Munich, Germany

Session 4 - Discussion

Future challenges in eye tracking for multimedia research (18 min)

 

Keynote

Title: Predicting eye movements in images and video: Recent progress and future challenges.


Abstract: As an overt signature of attentional state and current task, gaze behaviour has captured the interest of psychologists and neuroscientists for at least half a century. Significant recent advances in predicting gaze behaviour have come from applying modern machine learning techniques to gaze data. I will present a (biased) review of recent advances in comparing gaze prediction models on the one hand, and the models themselves on the other. First, I will discuss how adopting a principled probabilistic view of model comparison can resolve confusion in ranking models, before presenting a broader perspective on the difference between a fixation prediction model, the "maps" it produces, and the metrics used to compare models. Second, I will present a fixation prediction model that leverages the power of transfer learning to yield state-of-the-art predictions for gaze densities and scanpaths. Finally, I will present recent work on predicting gaze in video stimuli and the weaknesses of current datasets for this purpose. I will also provide some speculations on opportunities for advancement in understanding beyond prediction of gaze behaviour.


Lecturer: Thomas Wallis, Senior Research Scientist, Amazon Tübingen. Thomas has conducted research over a diverse range of topics in the field of perception and cognition, including low-level visual processing, crowding, eye movements and fixation prediction models, perception in people with low vision, comparing human and machine perception, and hazard perception in driving. He holds a PhD in Psychology from the University of Queensland (2010; with Derek Arnold). He was a postdoctoral fellow with Peter Bex (Schepens Eye Research Institute), a postdoctoral fellow with Matthias Bethge and Felix Wichmann (University of Tübingen) and a project leader in the DFG-funded research center for Robust Vision (University of Tübingen). Since 2019 he is a research scientist with Amazon in Tübingen.

 

Selected Publications

  1. M. Kümmerer, T. S. A. Wallis, and M. Bethge, “Information-theoretic model comparison unifies saliency metrics,” Proceedings of the National Academy of Sciences, vol. 112, no. 52, pp. 16054–1 6059, 2015.
  2. M. Kummerer, T. S. A. Wallis, and M. Bethge, “Saliency benchmarking made easy: Separating models, maps and metrics,” in Proceedings of the european conference on computer vision (ECCV), 2018, pp. 770–787.
  3. M. Kummerer, T. S. A. Wallis, L. A. Gatys, and M. Bethge, “Understanding low-and high-level contributions to fixation prediction,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 4789–4798.
  4. M. A. Pedziwiatr, M. Kümmerer, T. S. A. Wallis, M. Bethge, and C. Teufel, “Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations,” BioRxiv, p. 840256, 2019.
  5. T. S. A. Wallis, M. A. Dorr, and P. J. Bex, “Sensitivity to gaze-contingent spatial distortions in freely viewed movies,” Perception ECVP abstract, vol. 41, pp. 159–159, 2012.
  6. T. S. A. Wallis, M. A. Dorr, and P. J. Bex, “Sensitivity to gaze-contingent contrast increments in naturalistic movies: An exploratory report and model comparison,” Journal of vision, vol. 15, no. 8, pp. 3–3, 2015.

Scientific Chairs:

  • Dietmar Saupe, University of Konstanz, SFB-TRR 161 Project A05
  • Hantao Liu, University of Cardiff, Wales, UK, SFB-TRR 161 Cooperation with Project A05
  • Lewis Chuang, LMU Munic, SFB-TRR 161 Project C06
 

Program Committee:

Additionally to the chairs, the following people will be involved in the reviewing process:

  • Oliver Wiedemann, University of Konstanz, Germany
  • Jesse Grootjen, LMU Munich, Germany
  • Xin Zhao, University of Cardiff, UK

  

Contact Information

For questions and suggestions please contact Dietmar Saupe: dietmar.saupe@uni-konstanz.de.

 

Workshop Webpage

Twitter Thread


Event location:  Online digitally