AR Story Books

AR Enabled Picture Book

There is no doubt that technology has interjected itself into every piece of our lives especially young learners. So how we can keep children interested and help them fall in love with books even more? Introducing AR Story Books, an experience to allow further engagement with just a book and a smart phone.

Team

Lauren Owen

Software

Unity
Xcode

Duration

March 15, 2019 – May 10, 2019

Process

Inspiration

One third of children in the US start kindergarten behind in critical language and math skills according to a 2012 Brookings Report. I sought to exolore how augmented relaity could be helpful, given that I was relatively new to the medium. I was previously inspired by a previous project, Letter Learner, that aimed to aid in vocabulary development. I chose to ideate in this direction.

Mock-Ups

Ideation

I considered creating an AR app that parents can download to put digital alphabet in directly in their homes. Children can see each letter come to life as they interact with it learning what the letter is, its different sounds/phonemes, and ultimately help a child connect these sounds to print. I also considered creating an AR app that adds another reader to a child’s house hold. Children can turn to a page of a partnering book and a trigger will make the Reader appear on the smart device. This begins the reading of the page aloud by the now loaded character thus enhancing the reader experience with digital content, keeping the child engaged, and creating more opportunities for exposure to books and the positive results in doing so. I decided to go with the latter.

Implementation

I decided to load the page illustrations as animated videos using Unity and Vuforia. The rhyming alphabet book delved into various fictional monsters or creatures such as aliens, elves, and even Xing Tian. The story was written, and the illustrations were procured through a premium account I hold with FreePik, an image/vector licensing platform. After customizing the illustrations in Adobe Illustrator, I built out each page in Adobe Photoshop as different layer groups to ensure consistent formatting.

With jpg forms of these pages, I used Vuforia to create image triggers (see fig. 5). This allows a smart phone’s camera to recognize what is being viewed and prompt the set scenes to load. The results from Vuforia were loaded into Unity and each page’s scenes were separated. I chose to use a video overlay on the physical page and built that out through C# additions to the code backend accessible via XCode.

To create the animations and transfer them to video format, I used Adobe After Effects. The process was not burdensome as I was familiar with After Effects through previous products. I optimized each character or prop with the Puppet tool and used Move, Scale, or other effects as needed including the addition of sound effects retrieved through free audio libraries online. For the last page, Z, I chose to make a home video of Zuri appear. These videos were rendered and brought to Unity in their respective scenes. I then deployed to my smart phone to test and refine.