Presentation: Multi-Modal Input Design for Magic Leap

I’m the design lead for the Interaction Lab at Magic Leap.

We are part of the Product Org and formerly a content-focused wing of Systems. Our original charter included a mix of doing first turn-ons and system tests for new platform features. Over time we have shifted more toward being an incubation team, rapid prototyping experiences and looking for new ways of interacting with our platform using the many input modalities we support. One of our customers is developers, so we build tools, iterate on our SDK, and publish sample projects with documentation as part of “Magic Kit.”

We build a lot of proof of concepts to explore the possibilities of the medium. You will find us focused on questions like: what kinds of experiences and new products can we enable with our tools and features, and are our features and KPIs good enough to be able to build the kind of products we think are valuable?

We're a hybrid team composed primarily of former game developers. We’re customers of our own tech and, in the process, also try to make sure it’s usable for external developers. We’re very design and product focused and have a mix of engineers, artists, and designers who work very closely to discover the potential of our product and feed our findings to other teams and partners. Our motto is “Fail Fast!”

This year we finally pulled the curtain back on Magic Leap with Magic Leap One, a dev kit and our first public offering. We want to spread the word and get people aware of our platform and see how it's different from other mixed reality and augmented reality systems.

We want to show that as we move further into the spatial computing era, we can democratize computing even further, making it more accessible by embracing natural inputs and user expectations, including for people with disabilities. For instance, we can show how when you start combining input modalities in different ways, similar to how people use a keyboard and mouse together, you can unlock a lot of new potential use cases, particularly when done in a wearable untethered mobile form factor. We can combine your gaze with gestures and speech or other modalities to accomplish things not possible with existing mediums today.

Over time the trajectory of HCI has been to make computers conform more to humans. Early computers were very hard to use. You had to be an expert. One topic we’ll address is how we can improve existing interactions with computers through increasing intuitiveness and breadth of inputs, making functionality more natural and thus discoverable. Another is, because Magic Leap is wearable and mobile, it opens up new possibilities for tasks focused around spatial design, either in context of real physical spaces or involving simulating real objects at scale. There’s a lot of use there for a variety of industries including interior or industrial design, but also training, education, and more.

So much of our existing technology is used for escapism. I’m particularly aware of this, having spent the majority of my career making video games prior to joining Magic Leap. When you force people to focus on a screen, I think that’s what naturally happens. Escapism certainly has its uses, but too much can cause harm as well. I think that we’re seeing humanity having an extinction psychology moment right now where things have gotten really bad, so we’re doubling down on our escapism. I came to Magic Leap, because I think there’s great value in creating a new type of computing platform that keeps us grounded in the real world and lets us use it as the foundation for our digital experiences rather than taking humans even further out of their surroundings. Now, maybe more than ever before, we need to find ways to get people back grounded in reality. I’m excited for the potential we have to do that with this new medium.