Identifying Manipulated Images: New tools that analyze the lighting in images help spot tampering

Technology Review (MIT) ^ | March 17, 2008 | Erica Naone

[Stoat]

Identifying Manipulated Images

New tools that analyze the lighting in images help spot tampering.

By Erica Naone

True or false? The tool used above spots whether an image has been manipulated by modeling the lighting in the image based on an analysis of visible surfaces. To analyze an image, a user indicates the surfaces to consider using contour lines (shown above in white). The system checks for inconsistencies in the way that those surfaces are lit.
Credit: Micah Kimo Johnson, Hany Farid

Photo-editing software gets more sophisticated all the time, allowing users to alter pictures in ways both fun and fraudulent. Last month, for example, a photo of Tibetan antelope roaming alongside a high-speed train was revealed to be a fake, according to the Wall Street Journal, after having been published by China's state-run news agency. Researchers are working on a variety of digital forensics tools, including those that analyze the lighting in an image, in hopes of making it easier to catch such manipulations.

Tools that analyze lighting are particularly useful because "lighting is hard to fake" without leaving a trace, says Micah Kimo Johnson, a researcher in the brain- and cognitive-sciences department at MIT, whose work includes designing tools for digital forensics. As a result, even frauds that look good to the naked eye are likely to contain inconsistencies that can be picked up by software.

Many fraudulent images are created by combining parts of two or more photographs into a single image. When the parts are combined, the combination can sometimes be spotted by variations in the lighting conditions within the image. An observant person might notice such variations, Johnson says; however, "people are pretty insensitive to lighting." Software tools are useful, he says, because they can help quantify lighting irregularities--they can give solid information during evaluations of images submitted as evidence in court, for example--and because they can analyze more complicated lighting conditions than the human eye can. Johnson notes that in many indoor environments, there are dozens of light sources, including lightbulbs and windows. Each light source contributes to the complexity of the overall lighting in the image.

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

Hany Farid, a professor of computer science at Dartmouth College, who collaborated with Johnson in designing the tool and is a leader in the field of digital forensics, says that "for tampering, there's no silver button." Different manipulations will be spotted by different tools, he points out. As a result, Farid says, there's a need for a variety of tools that can help experts detect manipulated images and can give a solid rationale for why those images have been flagged.

Neal Krawetz, who owns a computer consulting firm called Hacker Factor, presented his own image-analysis tools last month at the Black Hat 2008 conference in Washington, DC. Among his tools was one that looks for the light direction in an image. The tool focuses on an individual pixel and finds the lightest of the surrounding pixels. It assumes that light is coming from that direction, and it processes the image according to that assumption, color-coding it based on light sources. While the results are noisy, Krawetz says, they can be used to spot disparities in lighting. He says that his tool, which has not been peer-reviewed, is meant as an aid for average people who want to consider whether an image has been manipulated--for example, people curious about content that they find online.

Cynthia Baron, associate director of digital media programs at Northeastern University and author of a book on digital forensics, is familiar with both Krawetz's and Farid's work. She says that digital forensics is a new enough field of research that even the best tools are still some distance away from being helpful to a general user. In the meantime, she says, "it helps to be on the alert." Baron notes that, while sophisticated users could make fraudulent images that would evade detection by the available tools, many manipulations aren't very sophisticated. "It's amazing to me, some of the things that make their way onto the Web and that people believe are real," she says. "Many of the things that software can point out, you can see with the naked eye, but you don't notice it."

Johnson says that he sees a need for tools that a news agency, for example, could use to quickly perform a dozen basic checks on an image to look for fraud. While it might not catch all tampering, he says, such a tool would be an important step, and it could work "like an initial spam filter." As part of developing that type of tool, he says, work needs to be done on creating better interfaces for existing tools that would make them accessible to a general audience.

[Stoat]

Please also see the Comments area at the article's original page where the Author as well as Dr. Neal Krawetz are answering reader questions.

[Fiddlstix]

Bump

[Stoat]

Ping

[SengirV]

I hope they take a historical look at all of AP’s evil Israel/pro-Palestinian photos.

[dead]

He was really there! I promise.

[anymouse]

Johnson says that he sees a need for tools that a news agency, for example, could use to quickly perform a dozen basic checks on an image to look for fraud. While it might not catch all tampering, he says, such a tool would be an important step, and it could work "like an initial spam filter."

Should be standard issue for AP and Reuters as well as Google News.

[evets]

[Stoat]

ROTFLMAO!!!!!

[tubebender]

Ping...

[beltfed308]

I hope they take a historical look at all of AP’s evil Israel/pro-Palestinian photos.

Green helmet guy will be "famous" if they do.

[Bobalu]

But a tool could be created to analyze an image and correct anything that these tools look at to detect fake images...thus rendering the tools useless.

[Old Professer]

In related news, NASA announced today that all images of moon landings have been archived and secured under the Homeland Security Act...

[stainlessbanner]

Good post Stoat

[evets]

[Stoat]

 

"snicker"

[ArrogantBustard]

No, Ms. Rice is not a Goa'uld.

Ms. Clinton ...

[Postman]

Yes, but another tool could then be created to detect the corrections.....usw ( a little German abbreviation there)

[snowrip]

Awesome.

BTW, the AP is screwed.

[Stoat]

But a tool could be created to analyze an image and correct anything that these tools look at to detect fake images...thus rendering the tools useless.

I suppose we could be looking at a tech war of sorts with competing R&D efforts....one side continuously working on technologies to identify manipulated photos and others trying to circumvent those efforts.  Sounds much like internet / computer security R&D in general.  In that arena, we have an ongoing war with neither side giving up and both sides claiming victories at one point or another, and all the while the end-user public is hoping that they are being adequately protected   :-)

[Stoat]

Good post Stoat

Thank you for your kind words  :-)

These technologies could conceivably put a real dent in the Left's favored spin methodologies....if they can't lie with images anymore, they haven't got much else  :-)