The Uncanny Valley and Camera Frame Rates

Warning: You might place this post firmly in the "bullshit rattling round in my skull" category. There's no real scientific basis for it, I'm not exploring it further, it just kind of occurred to me and I thought I'd write about it. It's a fuzzy idea in the truest sense.

Monday we went to the American Museum of Natural History and visited the Brain exhibit. No brain-based exhibit would be complete without a bit on the senses, and, as a visual artist I was interested to see the portion relating to vision. It was quite attention-grabbing and planted the idea for this post so I went for my phone and grabbed a few sloppy frames as reminders (really, please forgive the quality).

  

The picure on thel far left is a display where a bunch of lined-up spools of thread form a simple mosaic, but viewed through an orb (like the eye) the inverted image becomes a hazy Mona Lisa. The following panels describe how the brain interprets visual information and resolves it to something clear and familiar. Another display showcased blurred images of easily recognizable celebrities to illustrate how memory can be called upon to intepret even unclear images which increases the processing efficiency of the brain. Where things became really interesting to me was the caption on the above display.

"In the visual cortex at the back of the brain, Some [sic] [neurons] react to specific visual cues, like shapes and colors. Others pick up the signals and put the parts together to sense the whole. the entire process can happen in 1/50 of a second."

The figure of 1/50 of a second grabbed my attention because it's awfully close to the 1/48 sec standard shutter speed of a cinema camera. It seemed interesting how the standard duration of creating an image is so similar in a camera to what it is in the brain. The human visual system is more complicated than that though and it would be inexact to say that the visual cortex processes 50 images per second, but let's use that figure and, for simplicity's sake, think of the brain as having a frame rate. Since the standard frame rate of cinema is 24 frames per second our brains can be thought of as receiving movie images at about half the speed that they can process them.

I understand that the 24fps standard was created for technical reasons but it has, through decades of use, become part of cinematic grammar. To me, higher frame rates - such as 60 or even 30fps - are unattractive in cinematic conditions but perfectly fine while watching sports or broadcast television. Why the difference? I've heard it stated that 24 frames is more in tune with the rhythms of the mind and that watching movies at this rate in a darkened theater creates a dreamlike state. I'm not sure how accurate that is but it's certainly plausible.

All of this got me thinking about the Uncanny Valley, a concept used both in animation and in robotics to describe how the mind is repulsed by objects that look close to being real but not to things that look unreal. For instance, you can accept that Fred Flinstone is a human and you can accept that Fred Sanford is a human; however it's hard to accept, say, the 3D models from Beowulf as human. The mind just rejects it and it's a little creepy. Interestingly, there's a disparity in this effect between moving and still images.

What if the same holds true not just for objects and images approximating a realistic appearance, but also with images approximating realistic motion? Is it possible that motion pictures appear better when the intermitent motion is slower - fast enough for the brain to stitch images together and interpret them as motion but not so fast as to confuse the brain that it is actual reality? It's an interesting thought. What if I'm not put off by high-framerate sports because my brain accepts the subject matter as real whereas it expects fantasy from a movie? If the uncanny valley as it is applied to real-world objects is a defense mechanism then perhaps the same is true when it is applied to viewing motion photography. Is it easier for my brain to accept that the oncoming truck on the screen is not going to actually harm me because it interprets the slower signals as non-reality? Another thing: perhaps we feel this less in our living rooms & on a smaller screen - an environment where reality is more "present" than in a theater. While we may be intellectually aware of the difference in frame rates on a movie broadcast than one projected, it is less uncanny to us in our home. In the theater, though, everything is enlarged and the world around it is dark and my experience of reality is limited to what's going on on the screen. How we experience visual stimulus is conditional and may mitigate or exacerbate the effects of the uncanny valley, but I think the phenomenon may hold true for our experience of movies.

The point of this isn't really to bring up the frame rate debate. I just saw this exhibit and became interested in what it meant. I think it tells us as filmmakers that we should be deliberate and mindful of how we present our work and perhaps that it behooves us to know something about the brain beyond psychology.

I do have another interesting thought though. A couple of big name filmmakers are moving to shoot their 3D movies at 48fps and it's a contentious topic, but after writing this post I'm not so sure it's a big deal. If each eye receives 24 signals a second, the brain will receive 48 "incomplete" (in that each image lacks the stereoscopic dimensionality that the brain is used to) images per second. So, the question (in terms of my uncanny valley hypothesis) is, will 48 "incomplete" images seem as uncanny as 48 (or 60 or 30) "complete" ones and is it significantly worse than 24? If the objective of the visual cortex is to separate presented reality from expected reality then there may not be an issue. If that's the case then Mr. Jackson and Mr. Cameron are probably correct: the smoother presentation of stereoscopic footage at 48fps may indeed present a more canny 3D image.