Skip to comments.Blind "See With Sound"
Posted on 10/08/2003 4:00:29 AM PDT by foolscap
Michelle Thomas is learning to "see", not with her eyes but her ears.
Blind since birth, Ms Thomas is able to recognize the walls and doors of her house, discern whether the lights are on or off and even distinguish a CD from a floppy disk after only a week using a revolutionary new system.
She is "seeing with sound".
Developed by Dr Peter Meijer, a senior scientist at Philips Research Laboratories in the Netherlands, the system is called The vOICe (the three middle letters standing for "Oh I See").
It works by translating images from a camera on-the-fly into highly complex soundscapes, which are then transmitted to the user over headphones.
Watch the 'spikes'
A wearable setup consists of a head-mounted camera, stereo headphones and a notebook PC.
In total it costs about $2,500. The software is available as a free download.
He hopes that blind users will ultimately learn to mentally reconstruct the visual content of the live camera views, as carried by the soundscapes, so that they experience something akin to meaningful vision.
"Our assumption here is that the brain is ultimately not interested in the information 'carrier' (here sound) but only in the information 'content'," says Meijer.
"After all, the signals in the optic nerve of a normally sighted person are also 'just' neural spiking patterns. What you think you 'see' is what your brain makes of all those firing patterns."
Enabling users to get an audio snapshot of what is visually in front of them, The vOICe is taking a very different route from "bionic eyes" - retinal and brain implants.
It is non-invasive, offering a higher image resolution (up to several thousand pixels) and does not necessarily rely on the visual cortex.
The system's developer hopes ability will come with practice "Everything has its own unique sound and once you learn the principles involved you can know what you're seeing," says Thomas.
Right now brighter areas sound louder, height is indicated by pitch and a built-in colour identifier speaks out colour names when activated.
While it can't track fast cars or read small print efficiently, it does allow blind users to trace out buildings, read a graph and even watch television.
Comparing it in terms of difficulty to learning a foreign language, Meijer hopes that in the long run, users will become more "fluent" in the mental translation so that it becomes more like natural perception, without conscious effort.
Kevin O'Regan, of the CNRS (National Centre for Scientific Research) in Paris, France, and an expert in the area of sensory consciousness, is currently evaluating The vOICe.
This headset connects to a mobile PC He believes that if perfected, the software could at least partially evoke vision-like sensations in even the congenitally blind.
"The problem is that vision is a very high bandwidth system, and it's not clear whether we can achieve sufficient bandwidth via other modalities," he stated.
To suit user preferences, Blue Edge Bulgaria has developed a simplified but highly portable mobile phone version of The vOICe for the Nokia 3650 camera phone.
It is available as a free download at The vOICe site.
If you put two prisms in front of someone's eyes (like eyeglasses), such that the prisms invert up and down, the person will be confused and have trouble walking, as you might imagine.
But after 48 hours, they start to "learn", and after a week or so, they can walk almost normally. Their brain has adapted to the new way of seeing.
When you take them off, the person is disoriented again, and has to re-adapt back to normal.
Of course, recent reports of the "occular implant", (from the same MIT lab that designed the cochlear implant that now serves Rush so well), promises a future where certain types of blindness will be treated with a "bionic" eye.
This month's Scientific American has a cover story about artificial muscles. Special rubber like materials have been developed which contract when electrically stimulated. Ipso facto, amputees will eventually be fitted with prosthectics which will act as a full limb replacement. So now we have Steve's bionic eye , Steve and Jaime's bionic limbs and Jaime's bionic ear! It won't be long before we have a working Data as well!