The Uncanny Valley and Camera Frame Rates

Warning: You might place this post firmly in the "bullshit rattling round in my skull" category. There's no real scientific basis for it, I'm not exploring it further, it just kind of occurred to me and I thought I'd write about it. It's a fuzzy idea in the truest sense.

Monday we went to the American Museum of Natural History and visited the Brain exhibit. No brain-based exhibit would be complete without a bit on the senses, and, as a visual artist I was interested to see the portion relating to vision. It was quite attention-grabbing and planted the idea for this post so I went for my phone and grabbed a few sloppy frames as reminders (really, please forgive the quality).

  

The picure on thel far left is a display where a bunch of lined-up spools of thread form a simple mosaic, but viewed through an orb (like the eye) the inverted image becomes a hazy Mona Lisa. The following panels describe how the brain interprets visual information and resolves it to something clear and familiar. Another display showcased blurred images of easily recognizable celebrities to illustrate how memory can be called upon to intepret even unclear images which increases the processing efficiency of the brain. Where things became really interesting to me was the caption on the above display.

"In the visual cortex at the back of the brain, Some [sic] [neurons] react to specific visual cues, like shapes and colors. Others pick up the signals and put the parts together to sense the whole. the entire process can happen in 1/50 of a second."

The figure of 1/50 of a second grabbed my attention because it's awfully close to the 1/48 sec standard shutter speed of a cinema camera. It seemed interesting how the standard duration of creating an image is so similar in a camera to what it is in the brain. The human visual system is more complicated than that though and it would be inexact to say that the visual cortex processes 50 images per second, but let's use that figure and, for simplicity's sake, think of the brain as having a frame rate. Since the standard frame rate of cinema is 24 frames per second our brains can be thought of as receiving movie images at about half the speed that they can process them.

I understand that the 24fps standard was created for technical reasons but it has, through decades of use, become part of cinematic grammar. To me, higher frame rates - such as 60 or even 30fps - are unattractive in cinematic conditions but perfectly fine while watching sports or broadcast television. Why the difference? I've heard it stated that 24 frames is more in tune with the rhythms of the mind and that watching movies at this rate in a darkened theater creates a dreamlike state. I'm not sure how accurate that is but it's certainly plausible.

All of this got me thinking about the Uncanny Valley, a concept used both in animation and in robotics to describe how the mind is repulsed by objects that look close to being real but not to things that look unreal. For instance, you can accept that Fred Flinstone is a human and you can accept that Fred Sanford is a human; however it's hard to accept, say, the 3D models from Beowulf as human. The mind just rejects it and it's a little creepy. Interestingly, there's a disparity in this effect between moving and still images.

What if the same holds true not just for objects and images approximating a realistic appearance, but also with images approximating realistic motion? Is it possible that motion pictures appear better when the intermitent motion is slower - fast enough for the brain to stitch images together and interpret them as motion but not so fast as to confuse the brain that it is actual reality? It's an interesting thought. What if I'm not put off by high-framerate sports because my brain accepts the subject matter as real whereas it expects fantasy from a movie? If the uncanny valley as it is applied to real-world objects is a defense mechanism then perhaps the same is true when it is applied to viewing motion photography. Is it easier for my brain to accept that the oncoming truck on the screen is not going to actually harm me because it interprets the slower signals as non-reality? Another thing: perhaps we feel this less in our living rooms & on a smaller screen - an environment where reality is more "present" than in a theater. While we may be intellectually aware of the difference in frame rates on a movie broadcast than one projected, it is less uncanny to us in our home. In the theater, though, everything is enlarged and the world around it is dark and my experience of reality is limited to what's going on on the screen. How we experience visual stimulus is conditional and may mitigate or exacerbate the effects of the uncanny valley, but I think the phenomenon may hold true for our experience of movies.

The point of this isn't really to bring up the frame rate debate. I just saw this exhibit and became interested in what it meant. I think it tells us as filmmakers that we should be deliberate and mindful of how we present our work and perhaps that it behooves us to know something about the brain beyond psychology.

I do have another interesting thought though. A couple of big name filmmakers are moving to shoot their 3D movies at 48fps and it's a contentious topic, but after writing this post I'm not so sure it's a big deal. If each eye receives 24 signals a second, the brain will receive 48 "incomplete" (in that each image lacks the stereoscopic dimensionality that the brain is used to) images per second. So, the question (in terms of my uncanny valley hypothesis) is, will 48 "incomplete" images seem as uncanny as 48 (or 60 or 30) "complete" ones and is it significantly worse than 24? If the objective of the visual cortex is to separate presented reality from expected reality then there may not be an issue. If that's the case then Mr. Jackson and Mr. Cameron are probably correct: the smoother presentation of stereoscopic footage at 48fps may indeed present a more canny 3D image.

The First Artists, Herzog's "Cave of Forgotten Dreams"

Spoiler Alert: These are my thoughts on Werner Herzog's latest documentary and I find it difficult to describe my feelings without giving away some details of the movie. This isn't as bad as narrative spoilers where plot points are given away, but if you prefer to see this film on its own terms then stop reading now. Below the trailer I'm going to start writing.

First thing's first: let's get the 3D chat out of the way. I've written before about how I'm not bothered by stereography. Whether it's a gimmick or not is irrelevant to me so much as the filmmaker's reasons for applying it and how it's used to tell the story. I do not think the idea here was "come for the 3D, stay for the anthropology;" but even if it was, is that such a bad thing if the end result is a little enlightenment? I was interested to see how a serious filmmaker would apply stereo to his story and I wasn't disappointed. Yes, the element of spectacle is ever-present but spectacle and wonderment are at the heart of this documentary. In my opinion the 3D was elemental to this story, which is rooted in cave paintings found in the Chauvet Caves in France - the oldest cave paintings ever discovered and the first examples of human art. I say that 3D is elemental because these paintings are not applied to a flat surface but follow the contours of the cave walls and how you see them is very dependent upon the angle and depth at which you view them. I recall a painting of lions that seems entirely natural at the beginning of a shot but elongates as the camera tracks around the curved wall to reveal a different view which seems natural in its own way. In short, the feeling of presence and sensitivity to the grandeur of the cave was enhanced by the 3D.

It's difficult to say whether Herzog understood the movie he was making or not. The story, such as it is, is told in terms of repetition. We spend quite a bit of time staring at cave paintings, often returning to the most spectacular examples again and again. I think the intention is that each time we return with new eyes. At first we are lured into the overwhelming sense of wonder that one feels when first seeing the cave. It's all about astonishment. The second time we come, we do so as intellectuals - we've met the scientists studying the caves and they've begun to explain the importance of the paintings. We have an armchair anthropologist's comprehension of what the artists were and what world they lived in. From here we start to venture out into the world outside the caves so that we can be given a bit of context. We see demonstrations of how these people might have hunted, how they made music. We see their fascination with sex in the fetishization of the female form. We even get hints of religion and spirituality. We essentially begin to relate to the people who lived here 35,000 years ago. Now properly initiated, we can enter the cave one last time as the film crescendoes into a 10+ minute stereographic montoge set to an intense but simple and exotic score and we can experience the caves as our ancestors did.

Or at least I think that's the idea. Opinions may vary as to whether Mr. Herzong pulls it off or not. I think it's mostly effective and I quite enjoyed the journey. I could see the argument that the film goes on too long and that there's more movie than the subject requires. Even if that is the case, the filmmaker should be applauded for delving deeper into the subject than his medium might be able to support. His thesis is that this may be the very place where the modern human soul was born. I think this is probably true, or at least this is an example of the sort of thing that was happening during this time frame. Neandertahls were roaming Europe at this time and this is certainly where our species began its dominance. It is these primitive - though remarkably elegant - abstractions that began to make us Human. Creativity, self-realization, the recognition of the world around us as something more than just a habitat - Art - through this film Herzog seems to be telling me that it is Art that makes us human and I am glad to share his awe at this opportunity to reach back through time and stand among the first artists.

A moment that might be described as goofy or hokey shows an experimental archaeologist recreating a paleolithic lifestyle for himself. He wears furs as the prehistoric men would have at a time when all of this was under a glacier and he plays for us on a replica of an early flute. Though it seems a bit funny, this is the moment that for me - a music lover and long time musician - has the deepest impact. A flute from over thirty millenia ago is tuned to pentatonics! The song that he plays for us? The Star Spangled Banner. An instrument from a time long before civilization can be used to play the anthem of one of the most advanced nations on the planet now. Nothing can further demonstrate more to me that we today are echoes of the people who drew lions, horses, and bison on the caves of France so many ages ago.

3D is a Gimmick, So What?

3D is probably part of the future cinematic landscape and no amount of bemoaning that will make it not so. The people I hear complain most about stereo are people who love movies. But the future of cinema depends much less on the people who love cinema than on those who like movies. 

Click to read more ...

A Moment of Cognitive Dissonance

My immediate reaction upon learning that Avatar had won the Oscar for best cinematography was frustration. Don't get me wrong. I loved Avatar and and thought it was gorgeous, and though i'm not one of those guys that automatically decries anything that uses digital capture I do have a hard time accepting something with so much CG as a contender for best cinematography. A lot of what's being "shot" just isn't there and so my feeling was that the cinematography on a film like this is a far cry from the traditional direction of motion picture photography.

Or is it?

Click to read more ...