Resight Challenge

Augmented Reality Competition

Resight is an augmented reality (AR) spatial application company that uses artificial intelligence for mapping. They have developed a SDK that no longer requires pre-mapping to anchor and interact with content. The challenge asks competitors to use the SDK to build something new.

70 applicants submitted their ideas, and 15, including myself, were selected to proceed in the competition. Each competitor has access to mentors who are in the AR industry from leading technologist and developers to business executives and founders. They also have access to an online discord group to use as a resource and for the latest news and updates.

Using the UX design approach and with research I conduced on augmented learning, I created an educational augmented reality mobile application prototype and accompanying presentation and demo video. After review by judges against the other competitors, Language Learner won Best Idea and Presentation.

 

Sponsor

ReSight

Team

Lauren Owen

Duration

January 2021 – March 2021

App

The Test Flight for beta app testing can be found here.

Presentation

Demo Video

Process

01. Discover

Meetings and workshops to accurately determine client needs, goals, and limitations.

Meetings

An introductory meeting was held on January 7th, 2021 with Omri Stein, CEO and Co-Founder, and Daniel Yahel, Marketing Manager, of ReSight via Zoom. ReSight’s founding was explained as well as the challnege. Competitors are expected to prototype their ideas and walk users through the experience as a whole allowing a range of final submission formats from video demos to deployed mobile apps. Additionally, we are asked to reserach the mentors and share our selections.

Next steps include downloading the SDK and sample app to familiarize ouurselves with RSight’s technology and joining the online community as a technical resource and way to ask any questions.

Workshop

After the initial meeting and reviewing the terms and conditions of the challenge once more, I conducted a brainstorming session. With the intent of making an educational augmented reality experience based on my Master’s research on augmented learning and using ReSight’s SDK that does not require pre-mapping, I came up with numerous ideas and chose the following:

 

Language Learner

An AR language experience for children that brings a fluent teacher or objects to learn a language and interact with right into your home

Needs

Easy to use interface for target group of children

One to two levels of gameplay for proof of concept

Ability to speak into microphone and match diction or other measurement of success

Goals

Expand access to AR and learning opportunities for children

Create proof of concept by competition end

Limitations

Time – Competition ends February

02. Define

Using the information gathered from brainstorming and external resources, a brief Statement of Work (SOW) is created including project planning and requirements definition.

Purpose

The purpose of the mobile application is to achieve the following:
– Increase access to language building activities for children 10 and under
– Increase awareness of ReSight’s SDK
– Improve user satisfaction and usability of AR apps

Tasks

IT Infrastructure
– Plan infrastructure
– Deploy to app store

App Design and Development
– Collect assets (both English and Spanish)
– Storyboard and design different levels
– Incorporate accessible design practices
– Build out design and features
– Test app with user groups and collect feedback
– Refine based on feedback
– Launch and troubleshoot

STANDARDS & TESTING

Americans with Disabilities Act (ADA) Standards for Accessible Design
– Web Content Accessibility Guidelines 2.1 (view here)

User Testing
– U.S. General Services Administration (GSA) Technology Transformation Service Usability Testing Requirements (view here)
– Just In Mind User Testing Guide (view here)

Success

To measure the success of the app in respect to the aforementioned goals, the following metrics will be analyzed and compared to current analytic data of the site.
Metrics
– Number of downloads
– User feedback

03. Design

Using the SOW, work commences on creating styling, gathering assets, and sketching wireframes.

Screens

– Load
– Home
– Main Menu
– Game Play
– About
– Privacy Policy

Font

Header Font
– Bubble Bobble
Paragraph and Text Font
– Arial

Color Palette

– Multi-color/Rainbow

Assets

Image of bedroom with augmented reality elements such as numbers and foods with their respective Spanish translations

Sketches

04. Deployment

After meeting with mentor Avi Bar Zeev, and discussing the ideas thus far, emphasis was decided to be placed on the ReSight SDK’s mapping functionality. By doing so, the interactivity with language could be limitless as users build out their physical space with this digital realm. Removing the initial 2d and 3d assests and using real elements with geolocated user placed labels would not only add a more realistic element to the experience but truly overlay language learning on the physical world.

The possibility of completing all of these goals solo  within the time period proved difficult with external factors impacting the initial development timeline. I was able to create a proof of concept app using RealityKit and Xcode, and deploy to to Test Flight for beta testing. I also created a demo video and presentation.

Screens

User Flow

1. Walkthrough/Overview of instruction
2. AR starts and user instructed to walk around home and label rooms by clicking on room type
3. After labeling a room, user labels items within room by using the search feature and dragging tag to item location
4. Once room is complete, user selects language and begins language practice
5. Labels display translated name and clicking on label will play the pronunciation for practice and language skill development

05. Demo

Final mock-up and demo video

The Test Flight for beta app testing can be found here.

Demo