Accessible Tech: VR and Calcs for the Visually Challenged
Published by KermMartian 6 years, 11 months ago (2017-07-07T03:45:27+00:00) | Discuss this article

by Thomas Dickerson and Christopher Mitchell

Over the last few years, we have been happy to see an increasing amount of technology designed with accessibility in mind. One of the first examples that we saw was the Orion TI-84 Plus graphing calculator, a modified TI-84 Plus designed by Orbit Research for blind users. When we were at SIGGRAPH 2016 last year, we also VR viewing technology specifically tailored to nearsighted users such as those common in the tech community. We'd like to tell you more about each of those projects, and we'd like to open this up as a discussion about how you think technology can or should be made accessible to more users in the future.

The Orion TI-84 Plus Talking Graphing Calculator

We first encountered the Orion TI-84 Plus back in 2013 at our first T^3 conference. Back over four years ago, Shaun "Merthsoft" McFall described this initial encounter with an accessible graphing calculator: "It was quite an amazing device. When graphing it plays pitches for the Y value of the graph, the same while tracing. When the value is negative the unit vibrates. When on the home screen, it talks out the answer of the equation you enter, and you can put it in a mode where it tells you all the keys you’ve pressed. We were curious how easy it was to learn the keypad, and Ken [Perry, a visually impaired user and researcher] assured us that it’s very easy to pick up and learn, and that he had no trouble whatsoever." I eventually got an Orion TI-84 Plus of my very own to play with, and found it to be a fascinating tool.

The calculator itself is a classic TI-84 Plus, loaded with a custom OS version, with an extra module semi-permanently attached to the top. I won't exhaustively replicate the list of features we previously published in 2013, but the upshot is that it can speak the math and menus on the screen, as well as make tones and vibrate to represent graphed plots. To demonstrate other ways to visualize graphs, my Orion TI-84 Plus also came with a set of printed, embossed plots that you can feel to "see" the shape of a graphed plot. If you're looking for a full technical teardown, our friends at TI-Planet also posted a hands-on review of the Orion TI-84 Plus for your enjoyment and edification. Since we're a community of DIYers and hackers, they even explored clever ways to play music using the Orion module.

The Orion is certainly a technically impressive tool, and Orbit Research did a great job of creating a module that provides a lot of extra power with minimal hardware modification to the original TI-84 Plus. Perhaps its bigger claim to fame, though, is in its role as a tool for the visually impaired to explore math just as fully as the rest of us. The problem of being able to use a calculator as a blind or visually impaired person is probably something most sighted people would never even consider, but once you think about it, a whole host of problems pop up. First, there's the problem of just doing arithmetic with a calculator in the first place. Perhaps text-to-speech software and OSes' built-in calculator tools could do this, but that's certainly not a good solution for a classroom setting. Then, there's the problem of doing any kind of non-trivial math, working with lists and probability and statistics, and finally, the problem of finding a non-visual way to represent the shape of graphs and plots. The Orion module cleverly solves all these problems, allowing blind or visually impaired users to hear the keys that they're typing, the menus they're navigating, and even the shapes and axis crossings of the graphs they're plotting. Considering the Orion device, and trying to use it with my eyes closed, made me think about so many other technologies that must be challenging or even impossible to use without reasonably good eyesight. I applaud Orbit Research and TI for exploring this, and hopefully in the future more technologies (and at a lower price point) will look to solve similar problems.

VR for the Nearsighted
The Stanford Computational Imaging Lab has a long history of working on problems at the intersection of display technology and human ocular physiology, including the design of an eyeglasses-free display that could provide automatically vision-corrected images to users.

More recently, they've been working to address issues in virtual reality headsets (and other "near-eye displays"). Human vision builds on two separate types of eye movement. When your eyes swivel to look at a new object ("vergence"), they must also relax or contract the shape of the eye itself to alter the depth of focus. In a traditional near-eye display, the physical surface that a user is looking at remains in the same place, and so these visual cues become mismatched from one other, causing visual discomfort and fatigue, as well as potentially more painful symptoms.

At last year's SIGGRAPH we got to test their prototype VR display, using a number of different corrective display modes, including a "monovision" mode adapted from treatments for presbyopia, where one eye is used to focus exclusively on near objects, and one eye to focus exclusively on far objects. While that display, pictured below, is too bulky to be used as a wearable headset in the near future, we look forward to seeing how the technology progresses.

Accommodation-invariant Near-eye Displays

At this year's SIGGRAPH they'll be presenting more fully on those findings, but a short video has already been made available:

In a unification of their work on accessibility and physically-adaptive VR displays, they've also recently published a paper on correcting visual refractive errors in VR, using a hacked Samsung Gear VR, capable of moving the position of your phone relative to the lenses.