Spatial tasking on
Microsoft HoloLense
Interaction Design | Unity Development
Over a three-month research project, I designed Frontline-AR, an assistive task management experience for Microsoft HoloLens 1, developed for Daimler (Mercedes-Benz). This prototype was created at UbiMax GmbH in Germany
Problem Statement
As everyday tasks grow more complex, leveraging contextual spatial information can enhance efficiency in assembly and manufacturing.
Team Members
Ashutosh (Project & Software Lead),
Peter (Backend Lead),
Johannas and Artur (Software)
My Tasks
Prototyping , Interaction Design and Unity Development
Tools
Adobe Xd and Unity(C#)
​​
Field of vision
The inspiration to structure flow and contextual information come really close to design for web pages. However there are are few key differences.
1. Hololense 1 offers a narrow field of view of 35 degrees, so one need to design UI elements within that contraint.
2. Unlike single web page screen layout, in Hololense one could pin multiple windows in the spatial space which gives flexibilty on how to design flows
UI Prototyping
Both Adobe Xd, Principal app and Unity(C#) together to create the UI.
Home screen workflow
Microinteractions of the side panel
Tool Tip prototype. (Unity renders are used directly into Adobe Xd to visualise the flow)
Colours & Materials
HoloLens use color-sequential see-through RGB displays to render holograms. Holograms are created by adding light to the real-world light. This means that white will appear bright, while black renders transparent.
Some of the design learnings
1. Use dark colours for background and white for text.
2. Every colour is always slightly transparent.
3. Using full Black R0 G0 B0 will result in transparent colour.
4. Using transparent white outline around the 3D UI elements for enhanced visibility
Gaze Interaction and Sound Design
Sound design is often overlooked in mobile and web applications, but on HoloLens, it plays a crucial role in enhancing the holographic experience. In mixed reality, where interactions exist in 3D space, sound should reflect spatial depth and context.
For example, a simple button in different system states—
-
No Gaze
-
Gaze
-
Tap
—should not only have distinct visual feedback but also unique auditory cues to reinforce interaction
No Gaze to Gaze to Tap sound experience
Outcome
This is the compilation of the outcome of our work. Although designing on Hololense is still an unexplored teritory, below are some of the key design learnings from the project.