Council for Scientific and Industrial Research voice computing researcher and developer Dr Rynhardt Kruger has developed methods that could make it easier for blind or visually impaired people to read non-textual graphical information in electronic documents.

Blind or visually impaired persons access documents by using a screen reader that reads text out loud or displays it on a special hardware braille device. However, a large body of technical and scientific material remains inaccessible to them because mathematical equations and diagrams are represented as images and not as text that a screen reader can read.

Blind or visually impaired readers can use his methods to explore diagrams, geometric shapes and equations through sounds and as gestures on a touch screen or by text-based navigation. He developed them for his doctorate in Electronic Engineering at Stellenbosch University (SU).

Diagram content is played to users as a special form of sound from which they could interpret the content being explored. For example, in the case of a descending line from upper-left to lower-right on the screen, the resulting sound would be a tone descending in pitch, Kruger, who was born blind, explains.

“I devised two gestures that blind or visually impaired people could use to explore content on a touch screen. When one finger is dragged across the screen, the content directly under the finger turns into a sound. When two fingers are dragged across the screen, the line between the two fingers acts as a scanner and all content encountered by that line becomes a sound.”

Kruger says that, to convey an equation to readers, the different elements of the equation are represented as “rooms they can walk through”.

“The directions between rooms reflect the visual layout of the equation, so that, for example, the reader would have to move right in order to visit an element which is situated visually right of the element where the user is currently located.”

Kruger says his study showed that blind or visually impaired readers are able to identify the basic components of a diagram and to explore previously inaccessible equations using his approach.

“My methods allow blind or visually impaired readers to access graphical material directly and read equations and explore diagrams by just using a tablet instead of having to pay for costly hardware or conversion services.

“These methods will help to get an idea of a diagram’s shape where the shape itself is important, such as a previously unknown geometrical shape, and for accessing already published material,” he says.

He adds that he would like to see his methods built into a book reader application that will allow readers to study diagrams and equations directly from books.

Kruger became the first blind person at SU to obtain both his BSc honours and master’s degrees in computer science. For his Honours project, he developed a programme that allows blind musicians to study music notation. For his master’s degree, Kruger researched methods to allow blind people to access online virtual worlds.

“During my studies, it was not always easy to access the material in my textbooks. I realised that many technical documents do not adhere to existing accessibility standards for blind or visually impaired people,” he says.

“Currently, graphical material must first be converted to an accessible format before they can read it. The problem is that diagrams are still only accessible through textual descriptions or tactile diagrams, which are costly to produce. Textual descriptions are often inadequate to describe the complex relations between components of diagrams such as lines and rectangles or the contours of a geographical map.

“Also, when equations are available in an accessible format, they must be read in a linear fashion since this is how mathematics is expressed in current accessible formats. This hinders blind or visually impaired readers from accessing the visual layout of the equation,” he notes.