ABSTRACT
Cursors, avatars, virtual hands or tools, and other rendered graphical objects, enable users to interact with computers such as PCs, game consoles or virtual reality systems. We analyze the role of these various objects from a user perspective under the unifying concept of “User Representations”. These representations are virtual objects that artificially extend the users’ physical bodies, enabling them to manipulate the virtual environment by performing motor actions that are continuously mapped to their User Representations. In this paper, we identify a set of concepts that are relevant for different User Representations, and provide a multidisciplinary review of the multisensory and cognitive factors underlying the control and subjective experience of User Representations. These concepts include visual appearance, multimodal feedback, sense of agency, input methods, peripersonal space, visual perspective, and body ownership. We further suggest a research agenda for these concepts, which can lead the humancomputer interaction community towards a wider perspective of how users perceive and interact through their User Representations.
This is particularly true for indirect interactions, where input and output space are separated. In this context, the input space refers to the part or area of an interface where physical actions may be carried out to interact in a given virtual environment. The output space is the location where virtual objects are visualized and manipulated (e.g., a display or a screen). For example, when users activate a mouse to move a cursor on a screen in order to select a virtual object, the input space corresponds to the area on which the mouse is handled, while the output space is the computer screen where the virtual objects are visualized. However, not all user interfaces have this separation of input and output space. For instance, when users select a virtual object on a touchscreen (e.g., tablet or smartphone) directly through contact with a fingertip (Figure 1A), the input and output spaces are collocated. We refer to this phenomenon as direct interaction. In contrast, during indirect interactions users are not able to directly select virtual objects with their fingertips, but require a virtual representation (e.g., a cursor) to manipulate virtual objects. Thus, the cursor is the User Representation that enables actions with other virtual objects, in indirect interaction (Figure 1B)
Reconfigurations of the peripersonal space also occur for indirect interactions with computers, where the input and output space are always separated, as in the case of executing actions on a screen with a computer mouse (Figure 5B). For example, it has been found that, after active mouse manipulations, the auditory peripersonal space is expanded to also include the region around the computer screen, although it is far from and disconnected from the body
. Accordingly, it seems probable that interactions in immersive VR also lead to adaptations of the peripersonal space to the virtual tools that the user manipulates in the virtual environment. This may involve handling distant objects with a long virtual tool, or having an elongated arm that extends to allow access to objects beyond reach
7. Conclusion
In this paper we propose the concept of User Representations to analyze user perceptions of cursors, avatars, virtual hands, or tools and similar virtual objects from a common perspective. User Representations are virtual objects that serve as artificial extensions of the users’ physical bodies, enabling them to execute actions in virtual environments, which would otherwise be unreachable. Users can continuously control these User Representations through their motor commands, leading to a perceived sense of agency over the representation. These virtual objects can vary in several aspects, such as experienced visual perspective, appearance, and the sensory feedback provided. Further, the User Representation may be controlled through different input devices and mapping functions, which can affect the experienced sense of agency, remapping of peripersonal space, and in some cases the illusion of body ownership. In this context, we have shown how knowledge from psychology, neuroscience, and HCI can help us understand how users perceive their virtual User Representations. We are confident that our work will help researchers in HCI gain a deeper understanding of interaction through artificial user representations, as well as inspire future research on these topics. To this end, we propose a research agenda for these concepts that can provide the HCI community with a wider perspective on how the interaction of a user with a computer can be modulated by the design of their User Representations.
