Sensory Interfaces

The rising popularity of immersive digital media are influencing the lines of new research and creating new scientific domains. New approaches regarding immersive and interactive features form new narratives and designs of the user experience — all dwelling on a new type of information representation that is being conceived, which is necessary to improve the users’ interaction with synthetic environments.

Virtuality continuum, an arena of Virtual and Augmented Reality technology, is a new digital medium for the transfer of this information. Future possibilities of evolution of immersive media can be applied to all and any domain of living. All that navigates towards a suggestion that Virtual Reality as an ensemble of technological innovations can be proposed as a model of a new type of sensory experience that would represent a new evolutionary level of the interactive dynamics on the cognitive spectrum. This will help us to conceive new models of human expression, although there are many voices that claim divergence from human-ness.

Virtual and Augmented Reality are a new milestone in the way we define our relationship to reality. Immersive Information Technology has the potential of bringing about changes in societal organization and social reality of humans. Yet, what kind of changes are we looking at? Technologically speaking, there are numerous of how to approach the subject, and there is an abundance of resources dealing with this topic. In relation to the possibilities of new emergent realities never experienced and seen before, I want to conceptually address the potential of encompassing other human senses and channels of perception. This will create innovative languages of an entire new range of human perception that is almost unlimited. And while it is dependent on the technological revolution, I support the notion of it evolving as a natural language of the human sensory interface.

Situational Components of Extended Reality

The components of the extensions of human interactions in the immersive environment can be defined and situated as virtual objects, interactions, devices or narratives related to the virtual realm. In this new phase of integrating technology into human sensing, the perceptual field of reality widens and opens up. The next phase is to conceptualize these experiences within a digital format based on the information received. The experiences in the extended, wider field of reality is conceptualized via ASA — artificiality, simulation and alternative. This is the bottom line of the applied VR technology.

Virtual experiences and immersive environments, just like ‘natural‘ or physical environments, involve socio-cultural, symbolic representations that contribute to creation of new interactivity. However, the new spectrum includes a combination of organic and artificial interaction design — human-computer interaction, user experiences and user interfaces based on affective computing. In these new formulations we can find the evolution of analyses of human factors such as vision and motion tracking (eye movements, body parts movements etc.) and integrate kinetics of skeletal and muscular position sensing.

The main components of the current generation of Virtual Reality Systems focus on these key domains: software, hardware, technology, interactivity, application interface, cognitive spectrum and ethics [1]. These offer a taxonomic categorization for designing the Virtual Reality Systems and bring closer the formulation of standards of VR usability. They are also crucial to define other disciplines that have been affected by this progress, mainly fields dealing with human factors, medical research, ergonomics and of course, neuroscience.

You Are a Part of a (Multi-sensory) Story

“Virtual World — a medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation.” [2]

Virtual Reality plays a new, but significant role in how we understand and relate to our physical reality. It is a new way of interacting with information that is presented to us via a different medium and different combination of cognitive channels. It becomes multidirectional and embodied, and the challenge for the technological field, besides the product development, is to navigate these changes and design the channels for the information to enter the sensory field properly. Accompanied by new narratives and storytelling as a medium of new possibilities, humanity will explore Virtual Reality, as previously stated, not only in a technological sense. The technology will act as a companion to the new dynamics conceived by said technology that makes people a part of a new story. You are the story and the story is around you. These two sentences are the two basic pillars to defining Virtual Reality.

Definitions of VR evolve with the available technology and gradually integrate a higher range of human factors as well as possibilities of interacting with the environment. In the early 2000s, VR has been predicted to become “a high-end user-computer interface that involves real-time simulation (i.e. not precomputed, but computed as time is passing) and interactions through multiple sensorial channels — visual, auditory, tactile, smell and taste” [3].

VR evolves to the extent that technology evolves, and currently, we are at the stage covering the visual and auditory field, but not yet tactile, smell and taste. There have been experiments performed regarding tactility by producing artificial force fields simulating virtual 3D objects, but it has not yet been introduced as a standard VR feature. Yet it is an example of exploration of new natural user interfaces that include human sensing in the synthetic realm.

True Immersion

Constructing experimental immersive worlds based on data coming from the physical world is creating an analogy between both. It is about ‘transporting‘ the human presence into the synthetic environment by recreating a sensory input and developing a proprioception for interacting with the Extended Reality. Designing these interactions and classifying its components constructed by artificial intelligence is being addressed by the Spatial Presence Theory, attempting to define what it actually means to be ‘present in a determinate place‘. However, spatial presence must first undergo integration of theoretical frameworks of attentional processes and embodied cognition. These are the critical components in developing a true immersion based on actual presence that will be impactful in the next upcoming stages.

References:

[1] Earnshaw, R.A. (Ed.) Virtual Reality Systems; Academic Press: Cambridge, MA, USA, 2014.

[2] Sherman,W.R.; Craig, A.B. Understanding Virtual Reality: Interface, Application, and Design; Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 2002.

[3] Burdea, G.; Richard, P.; Coiffet, P. Multimodal Virtual Reality: Input-output Devices, System Integration and Human Factors. Int. J. Hum.-Comput. Interact. 1996, 8, 5–24.

Title credits: Ben Pearce