B04 | Adaptive Algorithms for Motion Estimation

Prof. Andrés Bruhn, University of Stuttgart
Email | Website

Andrés Bruhn

Prof. Dietmar Saupe, University of Konstanz
Email | Website

Dietmar Saupe

Lukas Mehl, University of Stuttgart – Email  |  Website

In contrast to existing approaches that mainly rely on a static model with fixed assumptions, this project aims at designing algorithms that are able to use previously learned knowledge adaptively if applied to novel scenes. This shall be achieved on the basis of a flexible variational model that allows both the integration of different types of prior knowledge and the development of suitable selection mechanisms that seek to identify valid assumptions by a preceding quantitative analysis of the scene. Here, activation weights and model parameters shall be estimated online.

Research Questions

How can typical problems and regularities in image sequences be detected reliably and how can they be quantified numerically?

How can this information be used to rate the difficulty of video sequences, e.g. for assessing benchmarks and for modeling algorithms?

To what extent can priors be learned for different types of problems and regularities?

To what extent can adaptive approaches be designed that are able to handle challenging real-world scenarios based on previously learned priors and decision algorithms?

Fig. 1: Adaptive motion estimation. (a) Overlayed input images. (b) Standard approach. (c) Adaptive integration of feature matches at erroneous regions. (d) Final result. Visualization given by color wheel in the center.

Publications

  1. H. Men, V. Hosu, H. Lin, A. Bruhn, and D. Saupe, “Visual Quality Assessment for Interpolated Slow-Motion Videos Based on a Novel Database,” in Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX), 2020, pp. 1–6, doi: 10.1109/QoMEX48832.2020.9123096.
  2. H. Men, H. Lin, V. Hosu, D. Maurer, A. Bruhn, and D. Saupe, “Visual Quality Assessment for Motion Compensated Frame Interpolation,” in Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX), 2019, pp. 1–6, doi: 10.1109/QoMEX.2019.8743221.
  3. D. Maurer, M. Stoll, and A. Bruhn, “Directional Priors for Multi-Frame Optical Flow,” in Proceedings of the British Machine Vision Conference (BMVC), 2018, pp. 106:1-106:13, [Online]. Available: http://bmvc2018.org/contents/papers/0377.pdf.
  4. D. Maurer and A. Bruhn, “ProFlow: Learning to Predict Optical Flow,” in Proceedings of the British Machine Vision Conference (BMVC), 2018, vol. 86:1-86:13, doi: arXiv:1806.00800.
  5. D. Maurer, N. Marniok, B. Goldluecke, and A. Bruhn, “Structure-from-motion-aware PatchMatch for Adaptive Optical Flow Estimation,” in Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol. 11212, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Springer International Publishing, 2018, pp. 575–592.
  6. D. Maurer, Y. C. Ju, M. Breuß, and A. Bruhn, “Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo,” International Journal of Computer Vision, vol. 126, no. 12, Art. no. 12, 2018, doi: 10.1007/s11263-018-1079-1.
  7. D. Maurer, M. Stoll, S. Volz, P. Gairing, and A. Bruhn, “A Comparison of Isotropic and Anisotropic Second Order Regularisers for Optical Flow,” in Scale Space and Variational Methods in Computer Vision. SSVM 2017. Lecture Notes in Computer Science, vol. 10302, F. Lauze, Y. Dong, and A. B. Dahl, Eds. Springer International Publishing, 2017, pp. 537–549.
  8. D. Maurer, A. Bruhn, and M. Stoll, “Order-adaptive and Illumination-aware Variational Optical Flow Refinement,” in Proceedings of the British Machine Vision Conference (BMVC), 2017, pp. 150:1-150:13, doi: 10.5244/C.31.150.
  9. M. Stoll, D. Maurer, S. Volz, and A. Bruhn, “Illumination-aware Large Displacement Optical Flow,” in Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2017. Lecture Notes in Computer Science, vol. 10746, M. Pelillo and E. R. Hancock, Eds. Springer International Publishing, 2017, pp. 139–154.
  10. K. Kurzhals, M. Stoll, A. Bruhn, and D. Weiskopf, “FlowBrush: Optical Flow Art,” in Symposium on Computational Aesthetics, Sketch-Based Interfaces and Modeling, and Non-Photorealistic Animation and Rendering (EXPRESSIVE, co-located with SIGGRAPH)., 2017, pp. 1:1-1:9, doi: 10.1145/3092912.3092914.
  11. D. Maurer, M. Stoll, and A. Bruhn, “Order-adaptive Regularisation for Variational Optical Flow: Global, Local and in Between.,” in Scale Space and Variational Methods in Computer Vision. SSVM 2017. Lecture Notes in Computer Science, 2017, vol. 10302, pp. 550–562, doi: 10.1007/978-3-319-58771-4_44.
  12. M. Stoll, D. Maurer, and A. Bruhn, “Variational Large Displacement Optical Flow Without Feature Matches.,” in Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2017. Lecture Notes in Computer Science, 2017, vol. 10746, pp. 79–92, doi: 10.1007/978-3-319-78199-0_6.