AR Story Books
AR Enabled Picture Book
There is no doubt that technology has interjected itself into every piece of our lives especially young learners. So how we can keep children interested and help them fall in love with books even more? Introducing AR Story Books, an experience to allow further engagement with just a book and a smart phone.
March 15, 2019 – May 10, 2019
One third of children in the US start kindergarten behind in critical language and math skills according to a 2012 Brookings Report. I sought to exolore how augmented relaity could be helpful, given that I was relatively new to the medium. I was previously inspired by a previous project, Letter Learner, that aimed to aid in vocabulary development. I chose to ideate in this direction.
I considered creating an AR app that parents can download to put digital alphabet in directly in their homes. Children can see each letter come to life as they interact with it learning what the letter is, its different sounds/phonemes, and ultimately help a child connect these sounds to print. I also considered creating an AR app that adds another reader to a child’s house hold. Children can turn to a page of a partnering book and a trigger will make the Reader appear on the smart device. This begins the reading of the page aloud by the now loaded character thus enhancing the reader experience with digital content, keeping the child engaged, and creating more opportunities for exposure to books and the positive results in doing so. I decided to go with the latter.
With jpg forms of these pages, I used Vuforia to create image triggers (see fig. 5). This allows a smart phone’s camera to recognize what is being viewed and prompt the set scenes to load. The results from Vuforia were loaded into Unity and each page’s scenes were separated. I chose to use a video overlay on the physical page and built that out through C# additions to the code backend accessible via XCode.
To create the animations and transfer them to video format, I used Adobe After Effects. The process was not burdensome as I was familiar with After Effects through previous products. I optimized each character or prop with the Puppet tool and used Move, Scale, or other effects as needed including the addition of sound effects retrieved through free audio libraries online. For the last page, Z, I chose to make a home video of Zuri appear. These videos were rendered and brought to Unity in their respective scenes. I then deployed to my smart phone to test and refine.