immersive memory experience in mixed reality

Totem Hero

Totem is an immersive experience that allows individuals to visualize memories in mixed reality, anchor them to objects in their home, and experience them with others.


Challenge Sponsor



Mai Bouchet
Zihan Cheng
Emi MacLeod
Raj Makker
Shelley Xia

Project Timeline

9 weeks

My Role

Gestural Interaction Design
Interaction Flow
Final Presentation




The Microsoft Design Expo is an annual design challenge for top design schools from around the world. This year, they posed the prompt "Intentional Design for Positive Cultural Impact in Mixed Reality" to seek out different potential applications for the HoloLens. 


opportunity framing

Given mixed reality's ability to overlay digital displays, objects, and information on the real world, we recognised its potential in adding new dimensions and layers of understanding in the real world. With that in mind, our team identified the home as both an interesting and fitting area of intervention since mixed reality allows for easy synchronous sharing and collaboration. With that, we wanted to create a product where putting technology in the home actually promotes human-to-human connection rather than replacing it. 

To better understand how best to intervene, we asked the question "What makes a home?". At the core, we came to define the home as the combination of people (relationships), collections (objects in our home), and memories (experiences). Totem lives in the intersection of these 3 crucial aspects of a home. 


solution - design concept

Totem leverages the memories that people already associate with their objects in their home and allows them to share them with others in mixed reality. 


Anchor memories to objects in the real world

Screen Shot 2017-05-27 at 11.42.28.png

Re-experience memories

Share or collaborate memories with others


STORYBOARDING as an ideation tool

Our team experienced a few creativity blocks during the ideation process and decided to each draw our own storyboards in order to flush our what we have in our minds. Through sharing our storyboards, we were able to arrive at a common theme and developed the following final storyboard:

After experiencing the shared memory together, the object is imbued with both the past but also their dsarecently-shared experience of going into mixed reality to revisit that memory. dsadsa illustration by Shelley Xia

After experiencing the shared memory together, the object is imbued with both the past but also their dsarecently-shared experience of going into mixed reality to revisit that memory.

dsadsa illustration by Shelley Xia


3d ui + gestural design

Experimenting with AR/MR UI 

object-anchored ui

Our team recognised the benefits of familiarity and low learning curves with 2D icons and tried to find a balance between the two options.

We chose a ring UI around the cup with a familiar swipe gesture motion to balance an unique 3D UI with an easily-recognisable gesture. 



hand gesture + selection feedback

For more precise interactions, we explored different methods of two-hand gestures to scale, move, set objects and environments. For quick exits and adjustments, we used one-hand gestures that are more intuitive and faster.

For user feedback, we imagined different methods of how an object lights up when it has been selected. 




interaction flow



Great for fast, scalable, and directional-driven input, we implemented this for features such as initiating the application, scaling totem from a globe to a fully-immersive experience, placing the memory into an object.


voice interaction

Voice and language input are imprecise but offer an infinite range of exploration and interaction, including the benefit of keeping the users' eyes focused on the experience and not an UI. For that reason, we implemented voice when the user is adding and editing to a memory. 


Audio Feedback

Audio is one of the most important design aspects to keep a user feel immersed and present in an immersive environment. Therefore, audio output and feedback increases from external cues to fully immersive 360 audio as the user increases their immersion. 


the experience

the what

Totem is a memory experience that is always initiated by the user. She can create a new memory, re-visit an existing memory, or share the memory or experience with others - all of which are anchored in an object. 

When the user selects an object, a highlight will first appear around the object and the a ring UI would appear. 


the how

To create a memory, the user can pull out the "add memory" icon. To experience an existing memory or to share them, she can pull out the other orbs. To add to the memory, the user can choose to add through her own digital archive (photos and videos) and/or pulling content from online. 

There are 3 level of the Totem experience. The user can choose to have a snowglobe-third-person view, or scale it larger to peek into specific parts of the memory, or lastly, completely immersing herself into it.

With each of these levels, we imagine that the audio will also match the scale - where the audio will become more immersive as they increase the scale. 


the why

Totem makes it possible for us to experience and share our memories in entirely new ways - making memories both tangible and possible to share in a physical space. It also creates a new psychological dimension for people to think and interact with their possessions differently - one in which we hope will make the home more meaningful. 


concept video


You can view our presentation slidedeck HERE.