exp_07

Motion Vision is a mobile app that captures the surrounding area by camera and highlights movement in real time. It can be used with or without an Augmented Reality (AR) headset.

car movement captures by the app

research questions

  1. How do non-human animals distinct movement from stillness and can humans learn from that?

  2. Modern cities are referred to as fast moving. If we erase everything from the field of view that is not moving, how much of the city is still visible?

development

  1. In our research on predators we focused on us watching a cat and on what scientists and philosophers have written about cat vision. Additionally we compared the Gear VR Headset and the Android Galaxy S7 in regard to their field of view to the eyes of cats and humans.
Cats: Sometimes we watch a cat from our workspace. When she is hunting, she places the upper part of her body on the ground and presses her hind feet against it. Sometimes she stays in that pose for many minutes and then very suddenly shoots out to catch prey. She either does not care about us or does not notice us.
Further research on cat vision taught us that cats have a faster refresh rate of the imagery they perceive than humans. Consequently, slow moving objects appear still to cats while they are able to see and track fast moving objects easily. Biologists found that the eye of a cat has more rods than cones, which enhances the cat’s ability to see at night, although it generally lets them see colors less vibrant than humans do. Cats have a horizontal field of view of 200°, humans of 180°.
In his essay The Animal That Therefore I Am (Derrida 2006, 84 in Animals) the philosopher Jacques Derrida describes an encounter with his cat. He outlines how the cat sees him naked before he sees himself seen naked by the cat and criticizes a philosophy that does not acknowledge that animals look at and address those who philosophize about them.
Gear VR Headset with Android Galaxy S7: The lenses of the VR headset Gear VR have a field of view of 96° for older and 101° for newer models (horizontally). When using the product for AR by accessing the smartphone’s camera, both lenses channel the camera image to the human eye thus reducing the field of view to that of the camera. For the Galaxy S7 a field of view of 78° is presumed. The logical result of this is that a human in AR has to move their head about twice as much to see their surroundings than they would without AR. In a setting with little light the Galaxy S7’s camera app displays a brighter but blurrier live image of the setting than the human eye does.
  1. When in use, the camera of a smartphone shows a live image of its field of view. However, what is visible on the screen is actually a stream of single images, taken but not stored by the camera, multiple times a second. We used this camera output – the stream of images – and built a programm in the game engine Unity that compares the current image in the stream with its predecessor. Iterating on the results, we did some tests to figure out how to visually differentiate movement from stillness.
What came first to our minds, when wanting to cause movement in our faces, was to blink extensively. Adult humans blink ten to 15 times per minute to keep their eyes lubricated and to disengage their attention. This means that self-caused movement is essential to human vision and happens semi-autonomic. Cats do not need to blink regularly, which is an advantage when hunting.
person blinking their eye person moving arms
  1. We figured it to be most interesting to reduce the similarities among sequential camera images to a point where they became invisible. Even though we found our tests with strong, opposing colors very powerful, we wanted the imagery to feel more organic and less out of a retro racing game. The aesthetics we favored is closely linked to hand drawn animation - a technique where single images that show successive steps of movement are presented with at least 24 images per second, a speed that makes the human viewer believe to see movement.
motion made by hands

try or catch

We went out and tested Motion Vision in the city of Kassel.

When standing still in the streets we saw moving cars, tramways, bikes and persons passing by. Looking up, we watched the movements of birds. Infrastructures disappeared as much as slow moving clouds, dancing leaves in the wind and still standing or slowly moving people or animals. When moving or turning our head these static objects became visible for the time of our movement.

When doing the experiment with groups of people, those without the headsets moved quickly to be seen by those having the augmented view. Some people told us they felt like in a movie or simulation.

car movement captured by the app car movement captured by the app a person using a stick is walking by

We also looked at animation, the illusion of movement. The cicada in Disney’s animation classic The Grasshopper and the Ants is almost completely in motion and when dancing, only their left foot becomes invisible for a few frames.

animated cicada dancing

conclusion

Motion Vision changes the way one moves and sees movement. That alone is an interesting experience. Likely, cats see their surroundings very differently than imagined by us, but their special focus on movement becomes more relatable for humans using Motion Vision. The app was created with a human agency in mind: To re-see and reevaluate the city as a system constantly in motion.

As discussed above, the cameras of modern smartphones like the Galaxy S7 have, compared to the human eye, a very limited field of view and a low resolution. Their field of view is even further away from the cat’s ability to see. On the other hand the camera allows better vision at night than the human eye and consequently makes Motion Vision as suitable for night as for day.

Words we use in this text like live image of a digital camera suggest that a camera shows its surroundings as if the camera was a hole in the smartphone through which an application can see. Actually, a smartphone really constantly generates this live image on a chip. The process of displaying generated, successive images in the camera app happens at a speed that has probably been adopted from film making and animation. Since the imagery of the app is visually linked to hand drawn animation, people thought that what they saw when using the app was clearly curated and framed in a way that an animated motion picture is. Despite that, they realized this curated, framed view draws its imagery from their surroundings and that their movement has influence on what they see. This led to the idea that the way humans draw information from their surroundings without Motion Vision is also already curated and framed.

Motion Vision can be understood as a first attempt to learn from non-human animals as highly specialized inhabitants of cities. Derrida acknowledges the cat’s gaze, but still only mentions the movement of his thoughts regarding this gaze. Another approach is to think the absence of language as something other than not interpretable, but to regard seeing and perceiving as thought or equal to thought. When opening up to animal inspired modes of perception we hope to blur the boundaries between humans and other animals that have been historically constructed. We suggest to rethink thought as a practice of seeing and perceiving, thus leading to our approach to re-see sight.