| Abstract: |
Introduction. When immersed in a Virtual Reality (VR) environment, the real objects surrounding the user could represent both obstacles to be possibly avoided to prevent injuries, both entities with which one could interact. In [1], we have proposed a preliminary system that is able to create a virtual scenario consistent with the 3D structure of the real environment where the user is acting. In this work, we present some new features of the system, with the goal of achieving a coherent interaction with objects present in the real environment and seen differently in the virtual one. As a use case, we present the identification of chairs, with the aim of sitting on virtual chairs in a VR environment.
Description of the system. Our system uses a rough 3D model of a room and builds a consistent VR environment, where virtual objects are put in the same position as real ones and occupy approximately the same size. The system is composed of 6 different steps: 1) Alignment between the 3D model and the real room; 2) Floor detection; 3) Voxelization; 4) Clustering; 5) Placement identification; 6) Swap. Moreover, there is an offline step, not present in [1], which consists in the creation of a dictionary of possible virtual objects to be used for the creation of the VR environments, considering the geometric description of clusters identified in the 3D model.
The procedure starts with the alignment of the 3D room model with respect to the real room, then we detect the floor, and then we convert the single mesh of the room into a 3D grid of small pillars.
In the clustering phase, we cluster the pillars that compose the grid, following predefined criteria. The method is recursively applied to every neighbor which satisfies the criteria. At the end of this process, we get a set of clusters representing furniture and objects in our room.
The following step is "placement identification'' in which we identify clusters positioned in the scene and we check if they represent specific objects. Here, as an example, we consider chairs. This is the novel and probably the most important part of the system. Indeed, depending on its results, the following substitutions will respect or not the semantic meaning of the objects originally placed in the scene.
To identify chairs, we started by observing that, after the clustering phase, pillars that composed a chair were grouped into two clusters of different heights: one cluster for the sitting and one for the backrest. So, to identify a chair, we consider every cluster in our scene that lays on the ground and does not have any other cluster above itself. Then, we cast four boxes that start from the considered cluster and move along one of the four cardinal directions (left/right/forward/backward) to check if there are other clusters in the reference cluster neighborhood. Then, if we find a cluster in the neighborhood, we check if the couple, composed of the reference cluster and the neighbor cluster, fulfills predetermined criteria, established by observing the different types of chairs present in our laboratory, university, and homes.
Currently, we are conducting a set of experiments to analyze how people sit down on VR chairs, compared with the standard behavior with real scenes, analyzing the biomechanics of the movement.
[1] Valentini, I., Ballestin, G., Bassano, C., Solari, F., and Chessa, M. Improving obstacle awareness to enhance interaction in virtual reality. IEEEVR2020 |