Identifying Manipulated Images

New tools that analyze the lighting in images help spot tampering.

By Erica Naone

True or false? The tool used above spots whether an image has been manipulated by modeling the lighting in the image based on an analysis of visible surfaces. To analyze an image, a user indicates the surfaces to consider using contour lines (shown above in white). The system checks for inconsistencies in the way that those surfaces are lit.
Credit: Micah Kimo Johnson, Hany Farid

Photo-editing software gets more sophisticated all the time, allowing users to alter pictures in ways both fun and fraudulent. Last month, for example, a photo of Tibetan antelope roaming alongside a high-speed train was revealed to be a fake, according to the Wall Street Journal, after having been published by China's state-run news agency. Researchers are working on a variety of digital forensics tools, including those that analyze the lighting in an image, in hopes of making it easier to catch such manipulations.

Tools that analyze lighting are particularly useful because "lighting is hard to fake" without leaving a trace, says Micah Kimo Johnson, a researcher in the brain- and cognitive-sciences department at MIT, whose work includes designing tools for digital forensics. As a result, even frauds that look good to the naked eye are likely to contain inconsistencies that can be picked up by software.

Many fraudulent images are created by combining parts of two or more photographs into a single image. When the parts are combined, the combination can sometimes be spotted by variations in the lighting conditions within the image. An observant person might notice such variations, Johnson says; however, "people are pretty insensitive to lighting." Software tools are useful, he says, because they can help quantify lighting irregularities--they can give solid information during evaluations of images submitted as evidence in court, for example--and because they can analyze more complicated lighting conditions than the human eye can. Johnson notes that in many indoor environments, there are dozens of light sources, including lightbulbs and windows. Each light source contributes to the complexity of the overall lighting in the image.

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

Hany Farid, a professor of computer science at Dartmouth College, who collaborated with Johnson in designing the tool and is a leader in the field of digital forensics, says that "for tampering, there's no silver button." Different manipulations will be spotted by different tools, he points out. As a result, Farid says, there's a need for a variety of tools that can help experts detect manipulated images and can give a solid rationale for why those images have been flagged.