Proving That Seeing Shouldn’t Always Be Believing

By CLAUDIA DREIFUS

 

HANOVER, N.H. — As Hany Farid sat in his office here at Dartmouth College on a recent morning, he fiddled with his laptop and cracked disconcerting little jokes. Caleb Kenna for The New York Times

‘I Think Like a Forger’ Using computer and mathematical techniques, Hany Farid can tell if the lighting is wrong, the fish isn’t that big or the celebrities weren’t really together. Caleb Kenna for The New York Times

“Don’t ever send me a photograph of yourself,” said Dr. Farid, head of the Image Science Laboratory at Dartmouth. “I’ll do the most terrible things to it.”

Dr. Farid, a 41-year-old engineer, is a founder of a subdiscipline within computer science: digital forensics. Most days, he spends his time transforming ordinary images into ones with drastic new meanings. Click, goes his mouse. Courtney Love has joined Grandpa at the family barbecue. Click. Click. Elvis Presley is on Dartmouth’s board of trustees.

The purpose of all this manipulation is to discover how computerized forgeries are made. Intelligence agencies, news organizations and scientific journals employ Dr. Farid’s consulting services when they need to authenticate the validity of images. Dr. Farid sells a software package, “Q,” to clients so they, too, can become digital detectives.

An edited version of two hours’ worth of conversation follows.

Q. Let’s start with some definitions. What exactly is digital forensics?

A. It’s a new field. It didn’t exist five years ago. We look at digital media — images, audio and video — and we try to ascertain whether or not they’ve been manipulated. We use mathematical and computational techniques to detect alterations in them.

In society today, we’re now seeing doctored images regularly. If tabloids can’t obtain a photo of Brad Pitt and Angelina Jolie walking together on a beach, they’ll make up a composite from two pictures. Star actually did that. And it’s happening in the courts, politics and scientific journals, too. As a result, we now live in an age when the once-held belief that photographs were the definitive record of events is gone.

Actually, photographic forgeries aren’t new. People have doctored images since the beginning of photography. But the techniques needed to do that during the Civil War, when Mathew Brady made composites, were extremely difficult and time consuming. In today’s world, anyone with a digital camera, a PC, Photoshop and an hour’s worth of time can make fairly compelling digital forgeries.

Q. Why do scientists need to know about this?

A. Because not long ago, researchers from South Korea had to retract papers published in Science because the photographs used to prove that human stem cells had been cloned were effectively Photoshop-cloned, and not laboratory-cloned. There have been other recent cases, too. And today, in science, more and more, photographs are the data. The Federal Office of Research Integrity has said that in 1990, less than 3 percent of allegations of fraud they investigated involved contested images. By 2001, that number was 26 percent. And last year, it was 44.1 percent.

Mike Rossner of The Journal of Cell Biology estimates that 20 percent of the manuscripts he accepts contain at least one figure that has to be remade because of inappropriate image manipulation. He means that the images are not accurate reflections of the original data. Rossner estimates that about 1 percent of the papers have some piece of image data that is downright fraudulent.

Q. Where does he get his figures from?

A. Mike has a full-time person who looks at every image supporting accepted manuscripts. Other biologists tell me anecdotally that many images in journals are regularly touched up to improve contrast or to remove little imperfections. The journals are, in essence, doing the same things fashion magazines do. Some of it is legitimate. In other cases, they are crossing the line.

Q. Are there policy changes that you think scientists should be considering?

A. I think it’s very hard to define inappropriate manipulation. Sometimes you can change 30 percent of the pixels in an image and it won’t fundamentally change anything. At other times, you can change 5 percent of the pixels and it radically changes meaning. I’m not a purist. I think there’s room for cropping, adjusting, contrast enhancement, but I want to know what was done. I think journal editors need to see the unadulterated, unretouched original images.

No. 2, the scientific community as a whole needs to come out with a well-thought-out policy on what is and isn’t acceptable when it comes to altering photographs. And this is something that must be refined, updated and changed as the technology changes. The journals are probably going to have to hire more staff. That will slow down the publication pipeline somewhat. But the cost of these scandals is too high. They undermine the public’s faith in science.

Q. You make software to detect forgeries. How do you design your programs?

A. I think like a forger. I spend a lot of time in Photoshop making digital forgeries to learn the tools and techniques a forger uses. We’ll make a composite photograph of two people and ask, “How do you manipulate this photograph to make it compelling?” By working backwards, we learn the forger’s techniques and how to detect them.

For instance, when looking at composites of two people, we’ve discovered that one of the hardest things for a forger to match is the lighting. So we’ve developed a way of measuring whether the lighting is consistent within various parts of the image. Lately, I’ve become obsessed with eyes. In a person’s eyes, one sees a slight reflection of the light in the room. So I’ve developed a technique that can take that little image of the reflection of light and tell us where the light was while you were being photographed. Does that match what we see in the image?

We also look at numbers. The pixels of a digital image are represented on a computer by numbers. Once you’ve altered an image, the numbers change. So we can analyze those pixel values for traces of manipulation.

Q. You consult regularly in legal cases. How is your work used in the courts?

A. I’ve consulted for the F.B.I., which sometimes uses images in prosecutions. They make surveillance tapes. At a trial, the defense might argue that the F.B.I. doctored the images. So how do you prove they weren’t doctored? That’s my job.

I’ve also been an expert witness in several child pornography cases. The Supreme Court in 2002 ruled that computer-generated child porn is protected under the First Amendment. So now in these cases, defense lawyers will sometimes argue that the images aren’t real. So far, I have only testified on the side of the prosecution. But I’ve been approached by defendants several times and I’ve told them, “I’ll work on your case, but I’m going to testify to whatever I find.” And in every situation, the defense lawyers said, “No, thank you.” In my opinion, that’s because they knew the photographs were not computer generated.

Q. What’s been the most interesting use of your software?

A. I sold a copy of it to a Canadian company that runs a bounty fishing contest. People send in photographs of fish they’ve caught. My program can check if the fish in the picture has been enlarged. We can prove whether or not the fish was really “THIS big!”