Posted on 02/27/2007 7:38:09 AM PST by Rio
Researchers at the University of East Anglia are about to embark on an innovative new project to develop computer lip-reading systems that could be used for fighting crime.
The three-year project, which starts next month, will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text. It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.
The university is teaming up with the Centre for Vision, Speech and Signal Processing at Surrey University, who have built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, who want to investigate the feasibility of using the technology for crime fighting.
The team also hope to carry out computerised lip-reading of other languages.
While it is known that humans can and do lip-read, not much is known about exactly what visual information is needed for effective lip-reading. Human lip-reading can be unreliable, even using trained lip-readers.
Dr Richard Harvey, senior lecturer at UEA’s School of Computing Sciences, is leading the project, which has been awarded £391,814 by the Engineering and Physical Sciences Research Council.
“We all lip read, for example in noisy situations like a bar or party, but even the performance of expert lip readers can be very poor,” he said. “It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill.”
Dr Harvey added: “The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence.”
As well as crime fighting there could be other potential uses for the technology, such as installing a camera in a mobile phone, or on the dash board for in-car speech recognition systems.
Another reason for developing computerised lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.
Dr Harvey said: “To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text.
“To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.
“The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far.”
All your thoughts are belong to us.
Think, what a devise like this could have done to all those karate and godzilla movies that use to be on Sunday afternoons years ago.
When this thing can make an accurate translation of Snoop Dog ranting about something, it will be done. Until then, it's in Beta.
I can just see all those hardened felons signing up for ventriloquism lessons...
Hey, Mohammed, those clever Brits might be reading our lips. Put your hand in front of your mouth when you speak.
Also, tell your wife... Oh, yeah. Never mind.
Heck, the HAL9000 could do that back in 2001.
Some people have waaaaaaaaaay to much time AND money on their hands.........sheesh.
Soon our thoughts will be read.
Does it speak jive?
...........remembering that Seinfeld episode where George uses Marlee Matlin's character to read the lips of someone at a party, there was a huge misunderstanding. Very funny. It won't be funny when this system misunderstands something.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.