My last project of the semester was a response to the concept of gesture. This is my second-ever installation-ish piece, and it was well-received. When someone moves in front of the camera, the motion is detected and represented on-screen as the spinning of hundreds of tiny discs. Faster motion makes the discs spin faster and change color more rapidly. The effect is even more interesting when using a projector to cover a whole wall with the spinning discs.

From watching people interact with this piece, I learned that everybody loves to see their actions interpreted and expressed in an alternate form. Although I find this project visually interesting, I was concerned about the fact that it doesn’t communicate any explicit information. That turned out not to be a problem from the user’s perspective, as everyone I’ve shown it to immediately starts waving their hands around, jumping up and down, and making all sorts of strange motions, becoming entranced by the patterns that “they” are creating on the screen.

After a couple minutes, though, the colors become such a hodgepodge that they cease to exhibit interesting patterns. So, following one of the great pieces of feedback I received, I added a timer that resets the grid once a minute. That reset encourages further experimentation and allows people to “take turns.” Users seem to get more invested in their motive experimentations when they can claim full ownership of the visual result, as transient as it is.

Watch a video here:

We reviewed our “gesture” projects in class last night, and while I think we all had some interesting ideas, no one envisioned an interface where your entire body could provide the input.

I just found out about this project called “You Move You Interact,” described as:

…an interactive installation, where one is supposed to build up a body language dialogue with an artificial system so as to effectively achieve a synchronized performance between the real user’s body and the virtual object itself. The project aims at exploring a spatial sphere, where the user/performer is invited to develop his own creative inspiration based on his own body gestures and movements. [ymyi.org]

Also interesting: It’s done using Processing.

“Coming Soon: Nothing Between You and Your Machine” is another excellent article in this Sunday’s New York Times. Although nothing new to anyone at DMI, it’s a great summary of recent steps toward redefining human-computer interactions, and it even gives a shout out to Processing.

I recently had the good fortune to visit the Design and the Elastic Mind exhibit at MoMA. It’s on exhibit until May 12, and I recommend that everyone interested in dynamic media and emerging technologies go see it. If you are reading this blog, that means you. (The image above is Jim Lambie’s installation on the first floor of the museum — literally, on the floor — and is not part of the design exhibit.)

I found out about the exhibit in a roundabout way. I was doing some research on the Make Controller Kit, an open-source hardware device that I’m considering for future projects. That site points to a number of great projects made with the kit, my favorite of which is Lightweeds by Simon Heijdens. Heijdens’ site mentioned that his work was on display at MoMA, so I went to the museum simply to view his project, with no idea that I’d also see a number of other phenomenal projects.

Lightweeds is a brilliant concept: Project lifelike “weeds” onto interior walls of a space, or what Heijdens calls the “artificial” space of a gallery. Collect live data from the environment outside the building (temperate, sunlight, wind velocity), and make the projected weeds grow and behave in response to that data, thereby establishing a connection between the natural and built environments. Here’s a close-up of one “weed” blowing in the “wind”:

Another highlight is the piece I Want You To Want Me, by Jonathan Harris and Sep Kamvar, who I recognized immediately as the same folks who brought us We Feel Fine.

I Want You To Want Me mines the Internet, looking for people who themselves are looking for someone else: a date, a partner, a spouse. The touch-screen interface represents each individual as a balloon which can be tapped to reveal something about who that person is (e.g. Mike, age 29, in Philadephia) and what he’s looking for (e.g. “a hot babe” or “a man my age I can really settle down with”).

The project is extremely engaging. Users’ innate voyeuristic instincts make the content interesting and relatable, and the balloons (data) can be manipulated through multiple views and filters, which enables the discovery of people who may want each other in real life. Both of these characteristics were inherited from the We Feel Fine project, but I Want You To Want Me employs far-superior graphics and a touch interface. (As an installation, it doesn’t have to struggle with the limitations of a web-based distribution platform.)

The absolute highlight of the exhibit, though, is Shadow Monsters, an installation by Philip Worthington. You enter a small room with a very bright light on the wall behind you, and your “shadow” is “cast” on the opposite wall. The shadow, however, is augmented in real-time to suggest how it would be seen in the mind of a child who’s been told to sleep, but can’t stop worrying about monsters under the bed. While your projected shadow grows horns, teeth, scary eyeballs and shaggy hair, the space is activated with growling, grunting, and gargling — the sounds of hungry monsters preparing to devour little children. My favorite moment was when I made an alligator-like shadow puppet with my arms, and it grew large teeth and spat in disgust toward the opposite wall. (There was audio for the spitting, too.)

Not to brag, but that last bit of computationally enhanced performance art drew a brief standing ovation from a crowd of onlookers. You see, with Shadow Monsters, there are the shadows, and there are the people casting the shadows, both of which could be considered performers, since they both contribute something to the space. And then there are the people outside the space looking in, watching the performance.

But, in reality, the onlookers are part of the performance, too. They unwittingly play the part of the little child, peeking out from under the covers, afraid and confused, unable to explain what’s real and what’s not, and unwilling to get out of bed (and into the space) until they can figure out what’s really going on. My theory is that I drew a few claps because I wasn’t afraid to “get out from under the covers” and really explore the system. (I was fairly confident that it would not actually eat me.) I raised my arms, stuck out my legs, made enclosed shadow-spaces (which is how to trigger eyeballs, I discovered), and did a number of other physical actions that would have gotten me kicked out of the museum, had I not been within that installation space. But I didn’t care, because I wanted to know how the algorithm worked, and besides, my attention was on the large projection in front of me, so I wasn’t thinking about how silly I looked until people started clapping.

I will end with the beginning. Most visitors to the exhibit won’t even notice the first piece, Genomic Cartography by Ben Fry. It’s a sequence of human DNA, visualized as extremely small, pink and gray letters, printed on a white wall. But the exhibit title and introduction are set on top of the piece, so it looks like just an interesting background.

I took away three profound insights from this exhibit:

1. The fact that the Museum of Modern Art, a venerable institution, staged this exhibit completely validated my decision to study dynamic media. This is absolutely the future, and absolutely what I want to do.

2. About half of the dynamic installations, including Shadow Monsters, were built using Processing. Awesome. Again, I feel like I am totally going in the right direction here, and I am inspired by the high quality of work that can be done with the Processing environment.

3. Most of the exhibited artists and designers are within a few years of my age. And they have stuff in MoMA. That gives me hope and also scares me. I better get crackin’.

I cannot recommend a visit to this exhibit highly enough. If you can’t make it to New York, the exhibit’s website is a nightmare to navigate, but at least you can read about all of the projects there.

As you can see, people seemed to have a good time at the ASCII art show on Saturday. I had a blast, and was really pleased with how my ASCII Photo Booth turned out.

The “booth” was weeks in development, and often took priority over my regular school work. But I learned a ton about Processing, worked with live video for the first time, and also figured out how to generate and print PDFs. Also, although I was half-expecting the application to crash at some point, it never did. What more could I have asked for?

Here’s what it looked like during installation:

It was fun watching people use it for the first time, and I got some usability-related insights that will help me improve future installations. But most people understood it right away. You sit down in the chair and see your image translated into ASCII text on the screen. Click the mouse, watch the countdown — 3… 2… 1… Smile! — the screen flickers for a moment, and a second later your image emerges on paper from a laser printer. Cool!

Update: Just posted this video of the photo booth in action:

One thing I observed is that the best images were created by the people who didn’t rush and took some time to experiment with the system. They would lean in closer to the camera, then farther back, watching the on-screen text regenerate in response to their motion. The final images were sharpest when the subjects sat completely still before and during the exposure. That felt appropriate, given that ASCII is old technology, originating from a time when computers were much, much slower and unable to process images at all. As with early photography, a clear image in ASCII takes time to develop.

I was happy to see people walk away with a physical artifact of the experience, in this case a photo of themselves or of a friend. I hope that one or two of those dynamically generated, original artworks will end up on a refrigerator somewhere. (If you had your picture taken, leave a comment below and tell me about your experience.)

More pictures from the evening below. The gallery sign (excellently designed by Colin, who curated the show):

JK’s ASCII video wall: