3D Indoor Scene Understanding with Scene Graphs and Self-supervision
Federico Tombari, Google and Technical University of Munich (TUM), Germany
Visualization in the Real World: Confluence of Visualization and Augmented Reality
Dieter Schmalstieg, Graz University of Technology, Austria
Data-driven Storytelling
Nathalie Henry Riche, Microsoft Research, United States
3D Indoor Scene Understanding with Scene Graphs and Self-supervision
Federico Tombari
Google and Technical University of Munich (TUM)
Germany
Brief Bio
Federico Tombari is a research scientist and manager at Google and a lecturer (PrivatDozent) at the Technical University of Munich (TUM). He has more than 180 peer-reviewed publications in the field of 3D computer vision and machine learning and their applications to robotics, autonomous driving, healthcare and augmented reality. He got his PhD in 2009 from the University of Bologna, where he was Assistant Professor from 2013 to 2016. In 2008 and 2009 he was an intern and consultant at Willow Garage, California. Since 2014 he leads a team of PhD students at TUM on computer vision and deep learning. In 2018-19 he was co-founder and managing director of Pointu3D Gmbh, a Munich-based startup on 3D perception for AR and robotics. He was the recipient of two Google Faculty Research Awards (in 2015 and 2018) and an Amazon Research Award (in 2017). He has been a research partner of private and academic institutions including Google, Toyota, BMW, Audi, Amazon, Stanford, ETH and JHU. His works have been awarded at conferences and workshops such as 3DIMPVT'11, MICCAI'15, ECCV-R6D'16, AE-CAI'16, ISMAR '17.
Abstract
3D scene understanding investigates the development of computer vision tools for new applications in the field of robotics, autonomous driving, augmented reality, architecture (among others). The capability of analysing an indoor scene by extracting its semantic components together with their attributes, such as pose and class, currently heavily relies on deep learning approaches. In this talk, I will illustrate some new directions in learning 3D scene understanding, focusing in particular on indoor scenes. I will first introduce the state of the art in SLAM and 3D reconstruction for indoor scenes, outlining some useful constraints that can be leveraged to improve reconstruction quality in the indoor scenario. I will then present some recent work aimed at using scene graphs and GCNs as tools on which to carry out inference for 3D scene understanding tasks, such as instance detection and scene retrieval.
Finally, I will focus on estimating the 6D pose of objects in scenes with cluttter and occlusion, and illustrate how self-supervised learning can be leveraged to improve pose estimation accuracy while relaxing the need for annotated data.
Visualization in the Real World: Confluence of Visualization and Augmented Reality
Dieter Schmalstieg
Graz University of Technology
Austria
![]()
Brief Bio
Dieter Schmalstieg is full professor and head of the Institute of Computer Graphics and Vision at Graz University of Technology, Austria.
His current research interests are augmented reality, virtual reality, computer graphics, visualization and human-computer interaction. He received Dipl.-Ing. (1993), Dr. techn. (1997) and Habilitation (2001) from Vienna University of Technology. He is author and co-author of over
300 peer-reviewed scientific publications with over 19,000 citations and over twenty best paper awards and nominations. His organizational roles include associate editor in chief of IEEE Transactions on Visualization and Computer Graphics, associate editor of Frontiers in Robotics and AI, member of the steering committee of the IEEE International Symposium on Mixed and Augmented Reality, chair of the EUROGRAPHICS working group on Virtual Environments (1999-2010), key researcher of the K-Plus Competence Center for Virtual Reality and Visualization in Vienna and key researcher of the Know-Center in Graz. In 2002, he received the START career award presented by the Austrian Science Fund. In 2012, he received the IEEE Virtual Reality technical achievement award for seminal contributions to the field of Augmented Reality. He was elected as a senior member of IEEE, as a member of the Austrian Academy of Sciences and as a member of the Academia Europaea. In 2008, he founded the Christian Doppler Laboratory for Handheld Augmented Reality.
Abstract
This talk will investigate the potential of visualization as a new use case for augmented reality. Visualization deals with the problem of helping the user in understanding big data. However, visualization has largely been confined to desktop computing and not really entered the realm of mobile computing. With augmented reality, we can transport visualizations to a real-world context. Embedding visualizations in the real world allows to leverage many fundamental human abilities, such as spatial memory or proprioception, which are largely dormant in conventional desktop environments. But using AR does not only allow us to utilize the space around us to support cognition, it also brings the visualization to the task location where physical action and reaction takes place. Since our human-made environments are increasingly connected and digitized, we can deal with the information right where it originates from sensors or devices scattered in the world. This talk will describe concepts, challenges and application of this important new research direction.
Data-driven Storytelling
Nathalie Henry Riche
Microsoft Research
United States
Brief Bio
Nathalie is a researcher at Microsoft Research since 2008. She is a team member of the EPIC (Extended Perception, Interaction, & Cognition) research group led by Ken Hinckley. Nathalie holds a Ph.D. in computer science from the University of Paris Sud, France; and University of Sydney, Australia. She published her research in over sixty articles in leading venues in Human-Computer Interaction and Information Visualization. She has received several awards for her research and is involved in the organizing and program committees of major visualization and human-computer interaction conferences.
Abstract
Data visualization is a powerful medium to makes sense of large amounts of data and communicate insights gained from analyses to a general audience. Research in the field of information visualization aims at designing interactive visual interfaces to augment human cognition for exploring and communicating with data.
In this talk, I will present our latest research efforts in the field of data-driven storytelling. Stories supported by facts extracted from data analysis (data-driven storytelling) proliferate in many different forms from static infographics shared on social media to dynamic and interactive applications available on leading news media outlets. I will present research shedding light on what makes visual stories compelling and share insights on how to empower people to build these experiences without programming.