We invite you to the following workshop as part of the SFB/Transregio 161 at the University of
Crowdsourcing Visual Quality
December 03 - 04, 2018 organized by Vlad Hosu and Franz Götz-Hahn (University of
Room to be announced.
Please let us know
latest until Tuesday, November 18th, 2018 by email whether you wish to attend.
This extension to last year’s in-depth crash-course on crowdsourcing is aimed at both
teaching doctoral students new to the topic of crowdsourcing to efficiently conduct crowdsourcing
studies as part of their doctoral thesis work and conveying new insights on crowdsourcing to those
doctoral students who already participated last year. The seminar will cover the whole process of
performing an experiment and introduce all the analysis and auxiliary tools we have successfully
applied. Starting with an introduction to the topic, participants will be shortly introduced to the
basic experimental procedure. Subsequently, students will get to design a small scale crowdsourcing
study by themselves or extend a previous study to incorporate some of the information learnt. In
the process, they will be guided around the pitfalls of experiment design, as well as given a
spectrum of handy tools that simplify the process. Particular attention will be focused on
crowdsourcing subjective quality attributes and annotations, such as qualities of
non-photorealistic renderings and visualizations, or annotating artifacts in such renderings.
Additionally, participants will experience crowdsourcing from the crowd worker’s position, to
better understand the crowd. One invited speaker will share their insight into the dynamics of
crowdsourcing and experimental procedures. Finally, the results of an example crowd-study will be
analyzed, and any remaining questions will be resolved in an open discussion.
Matthias Hirth – Julius-Maximilians-Universität Würzburg, Würzburg, Germany
1. Crowdsourcing in an enterprise setting
2. Best practices for quality of experience
3. Mobile crowdsensing