3D User Interface Design — Part 3 - Human Interface and Evolving Guidelines
The best approach to 3D interfaces is a matter of debate. We can approach it systematically or artistically. All depends on our professional background, preferences and the general outlook. The best way to go about it is to combine them: artistic inspiration can spark fruitful systematic exploration, while discoveries made during structured research can be artistically applied. Ultimately, both of them are novel and effective practices and principles for their respective designs.
The Evolving Ideal
Interestingly, there is a long-standing standard for ideal interfaces available. The Macintosh Human Interface Guidelines to the API of the Mac OS originally offered guideline documents and interface software tools that served as basic building blocks for user interfaces. It is no surprise that these interfaces became so desired and popular.
They cover things such as:
• Layout – How controls should be arranged on screen and their relative sizes
• Controls – What types of buttons, menus, dropdowns, etc. should be used and how they should look and behave
• Text – Fonts, styles, capitalization, etc. for labels, buttons, menus, etc.
• Icons – Design guidelines for icons
• Color – Recommended colors and color palettes
• Interactions – How users should interact with controls (clicks, gestures, etc.)
The purpose of the guidelines was to promote consistency across Macintosh applications and a cohesive user experience. By following the guidelines, developers were (and still are) able to create applications that feel native and intuitive. The guidelines have evolved over time as has the Mac OS, sharpening the edges of the user interface standards. Earlier versions from 1984 focused more on the original aqua graphical interface, while later versions incorporate changes like support for gestures, trackpads, touch screens, etc. In 1988 Apple achieved some interface consistency as the ecosystem grew. The guideline version 3.0 was published in 1994, followed up by version 4.0 in 1999 that included new technologies such as drag and drop. In 2012, the version 10.8 already covered gestures on trackpads and touchscreens, full screen apps, and auto layout.
In recent years, Apple has incorporated more sophisticated 3D technologies into macOS and iOS, including perspective zoom transitions, 3D Touch interactions on iOS devices, improved 3D rendering in apps like Photos, Maps, etc., and finally support for VR and AR technologies.
But since we are still in early development, the recommendation to match the human interface guidelines is for the role of 3D effects to be restrained and clear. We should only use 3D effects to enhance usability, not for their own sake, keep 3D effects subtle and consistent, and avoid complex 3D interfaces that are hard to navigate. Complex 3D interfaces, however, need to evolve, because they are needed in XR.
Designing 3D interfaces has not yet reached the same level of development as that of desktop user interfaces, which have more established paradigms such as the WIMP paradigm. Much research has been done into 3D interaction techniques and associated human factors issues, but there is no unified vision to how these can be applied in interface design. The vastness of the 3D design space combined with ever-evolving input/output technology restricts progress, necessitating designers to continually innovate interaction techniques based on existing experience and knowledge.
Progress Towards 3D UI Guidelines
To develop a successful 3D interface, developers must first use existing research on human factors in computing systems, which is ideally followed by applying research-based interaction techniques and ideas. Then they can let their creativity run wild. But from experience, they should aim for as much simplicity as possible, which will help them invent interfaces and new interaction techniques. That can then be combined with existing design strategies and models for designing 3D interfaces. Ultimately, standard design models and strategies for 3D interface design must be utilized. Basic human factors principles that can easily be applied include research on two-handed interaction, constraints, sensory feedback, and multimodal interaction. Additionally, general 2D interface design rules – simplicity, consistency, error prevention – need to be taken into consideration.
Techniques for inventing 3D interfaces can be divided into two categories: realism-oriented and magical. Realistic approaches include replicating users' real world physical interactions, taking inspiration from movies and architecture, converting 2D user interface elements into 3D widgets, among other things. The magical approach focuses on relying on cultural clichés and metaphors -- e.g. the flying carpet and magic wand metaphors -- or willingly disregarding users' expectations.
These techniques allow us to design short-term, and often ad hoc solutions, but they do provide a foundation for scientific scrutiny based on which we can analyze, categorize and evaluate proposed ideas and techniques, leading to the systematic approach to interface design.
Systematic Philosophy – Rapid Testing and Evaluation Strategy
As part of the systematic approach to interaction design, we study users' tasks, existing interaction techniques, and characteristics of the user, the environment, or the system are considered. Rather than making sudden leaps in performance, this approach is slow and methodical.
Systematic philosophy's use of taxonomies is fundamental to its purpose. By breaking down tasks and techniques into categories, we gain a clearer perspective, making it possible to create new designs and understand the existing ones better, just like Apple did in the 1980s. One type of taxonomy, which organizes the task in a hierarchical structure before detailing technique components for individual subtasks, can be employed in guided design. This often contains software for prototyping and rapidly testing out interaction techniques.
A second important factor in the organized strategy is evaluation. Although most view design and assessment as distinct ideas, they are actually very intertwined. The recurrent designing method used in general HCI situations can be applied to VEs too. Evaluation of a design leads to alterations within that design, which then can be tested again. In 3D communication, the fundamental evaluations of communication techniques are still significant, yet formative and summative usability assessment of VE applications must also be included.
Thoughts for Designing Large-scale Virtual Environments (VE)
The importance of 3D interaction research is only going to expand as VEs become more intricate and extensive. I provided a brief overview of the field regarding effectual 3D user interfaces. The concepts of classic UI design can be applicable to 3D UIs in many cases, however 3D communication features unique characteristics that need new design requirements, which were discussed above. Several overarching principles for 3D user interface design can be provided given the current state of the art in this area. These are:
Whenever productivity and efficiency are crucial, use "magic" interfaces instead of "natural" ones. Natural interactions should be used when a replica of the physical world is required.
It is important to choose interaction techniques based on the requirements of the application – the same set of techniques will not work in every application.
Whenever possible, limit the degrees of freedom for input, and provide physical or virtual constraints to assist the user.
Provide the user with structure and support to manage complexity as a result of whole-body input, multimodal I/O, and novel input devices.
Conclusion
Instead of concluding with a statement, I will conclude with questions. There are still many unanswered questions and research issues in the field of 3D interaction. Some of the most important unresolved issues are:
1. Will a standard for 3D interfaces ever be practical?
2. How can VEs best support complex system control tasks (e.g. very large sets of menu items)?
3. How can we determine which 3D interaction techniques work best for a given
application?
4. How do various 3D interaction styles affect the user's sense of presence?
I imagine – what was it like being in the 1980s trying to come up with guidelines for 2D interfaces? That is what our 2020s are for 3D interfaces. Although exponential and much quicker development is expected, it doesn’t take the excitement and wonder away from us who can be a part of this task.
Recommended resources:
The Macintosh Human Interface Guidelines
GNOME Human Interface Guidelines
OLPC Human Interface Guidelines