For viewers, it’s quintessential, over-the-top Hollywood action. For scientists, it’s a window into the human brain. At a recent event here hosted by the Academy of Motion Picture Arts and Sciences, neuroscientists and cognitive psychologists got together with filmmakers to discuss what both groups have learned—the scientists through painstaking experiments and analysis, the filmmakers by intuition and experience—about the mechanisms of attention and perception.
In the Iron Man 2 sequence, for example, people are remarkably consistent in where they direct their gaze. Tim Smith, a vision scientist at the University of London, presented eye tracking data collected from 75 people as they watched the clip on a flatscreen. A camera tracked their eye movements, and software created a frame-by-frame heat map. When Smith played the clip with the eye-tracking heat map overlaid, the red hot spot tightly followed the action—people focused on the dueling superheroes, especially their weapons and faces, and on the car parts bouncing around (to see the clip, scroll down to the first video here).
Jon Favreau, who directed Iron Man 2, was onstage with Smith as he presented the clip, and seemed fascinated by it. “Everything you’re looking at is real, and everything you’re not looking at is fake,” he said.
“We’re constantly calculating where we think the audience’s eye is going to be, and how to attract it to that area and prioritize within a shot what you can fake,” Favreau said. “The best visual effects tool is the brains of the audience,” he said. “They will stitch things together so they make sense.”
What you can’t fake, Favreau said, are faces and physics. Favreau is working now on an adaptation of The Jungle Book, and he says almost everything is CGI except the faces. Faces are just too hard to fake convincingly, he said, even with sophisticated motion capture systems designed to capture every eye blink and facial twitch.
“It’s the same with physics,” Favreau said. In the Iron Man 2 scene, his special effects team created replicas of Formula 1 cars with the same weight and dimensions as the real thing and launched them with hydraulics or air ramps to create the flying, cartwheeling spectacle you see onscreen. “You get a tremendous amount of randomness in the way these things bounce and tumble and roll as they hit the ground and interact with each other, and that creates a sense of reality,” Favreau said.
While film makers intuitively understand things about visual perception and attention, scientists are trying to understand these things at a more mechanistic level, Smith said. He and other scientists want to know, for example, how the brain constructs a fluid perception of the visual world. “Visual perception feels like a continuous stream, but it’s not,” he said. What actually happens is that we look at one thing at a time, taking in a bit of information here, then moving our eyes to take in a bit of information over there. Then, somehow, amazingly, our brain stitches all those bits together to create a seamless experience.
That’s basically what film makers have been doing for a century, Smith said. It’s the essence of their craft. “As a psychologist, I’ve realized I can learn a lot from film makers.”
This is the first in a series of stories about how scientists are studying cinema for clues about the nature of perception, and how the science might aid film makers as they pursue their art.
source: wired.com By Greg Miller