Work at SCIM with Jody Sperling

Renowned dance artist and Loïe Fuller expert Jody Sperling is visiting this week, teaching workshops and helping with the “Master Dancer” VR app development. We have been working on real-time control of avatars in Unreal Engine, mocap for “AI Loïe, and kicking the tires on the real-time AI video generator programmed by Jake Metz at the SCIM lab in the UIUC Engineering Library. Exciting work!

Experimenting with AI generated video in SCIM lab.

Work proceeding on low-cost body trackers

Sean Jasin, an Electrical Engineering major, along with his fellow student Luke Liu, has been working on a new, low-cost solution for full body tracking in VR. This will help us to integreate more of the body into our movement explorations with the movement games in Master Dancer.

This video short shows the data coming from the sensor into the computer via WiFi. The cable that connects the breadboard to the computer is there simply to provide power, which eventually would be supplied via a small li-po battery:

https://youtube.com/shorts/pq5Mw_jIMEY

Master Dancer VR project underway

With the help of a grant from the University of Illinois Research Board, U of I Dance Professor John Toenjes has been able to work with a talented team of students to begin creating a Virtual Reality dance adventure game tentatively called “Master Dancer.” Not only will the game itself be used to reach out to other communities, but the technologies themselves used in developing this game are also going to be used to reach out to the community around the university. For example, Professor Toenjes is partnering with the HipHop Express project to take the cutting-edge motion capture suits used to create dance animations into underserved communities to expose them to, and excite them about, the possibilities of new technologies for cultural expression. This project is supported by the U of I Research Board, the Grainger Engineering Library IDEA Lab, and the Siebel Center for Design.

A playlist of videos of the project is viewable by clicking HERE.

Detailed Description of the Master Dancer Project

Much is being written about the centrality of computer games to contemporary human experience. In Reality is Broken,[1] author Jane McGonigal presents staggering statistics of the number of game players in contemporary life: even in 2011, 3 billion hours per week were spent gaming world-wide. As she writes, “Games, in the twenty-first century, will be a primary platform for enabling the future…[our] neurological and physiological systems—our attention systems, our reward center, our motivations systems, our emotion and memory centers—are fully activated by gameplay.”[2] Gamification of dance and the other performing arts can promote deeper involvement, understanding and appreciation through direct interaction with these art forms, and gives the creator new ways to directly affect the audience’s attitudes and feelings towards issues being dealt with in contemporary performance.

In order create art that is also a game, the artist must ask themselves: what goal can I construct for my performers to reach? How do I construct rules that make the game compelling and give it structure? What is the feedback system or systems that the performers and/or audience will rely upon to feel like the art is progressing along a line of inquiry? Lastly, how do I get the audience to want to participate?

A technology that is akin to live performance is Virtual Reality, which has an immersive quality like the experience of being in a theater with the house lights out and the theater brightly lit. But VR has the interactive capabilities of a game controller, which makes it a ripe technology for just the kind of game/art integration described above. Therefore, I have embarked on a quest to make art/games in Virtual Reality.

Description of “Master Dancer”

Master Dancer is just starting to be under development. The goals of the Master Dancer game are: 1) to instruct users in some basic movement energies and then to get them to use those new skills to move in their own creative way, and 2) to be exposed to some historical dance information. The movement energies are based upon Laban Movement Analysis principles, but they are more generalized than the eight effort actions. (The video example below still uses the Laban term “slash,” but the finished version will remove that in lieu of the more general term “directed forceful movement.”)

The basic movement energies are taught through movement games that are scored and these scores compared to those of other users. The motivation to continue in the game is in the story, as follows. The famous dancer Loïe Fuller (1862-1928) is found in an old, decrepit theater. She asks the user to help her restore the theater and her dancing abilities by becoming a better dancer. This is accomplished through training in the movement energies by playing action games. After the user finishes each training game in a “training theater,” they re-enter the theater lobby to find the Loie is a little younger, and the theater a little brighter. They are asked to dance a creative improvisation based upon the skill they just learned, and this dance is recorded in movement trails—a “living 3D sculpture.” This living sculpture will then appear on the lobby pedestal (seen behind Loïe in the opening), an elaborate sculpture eventually constructed of movement trails of different colors and types based upon the energies of the different movements required to master the dance skill games. Once all the dance skills are mastered, a final dance with a re-invigorated Loïe in the fully restored theater is the reward.

In the lobby, playing on the jukeboxes at the entrance to each game, are videos of dance types that are made up primarily of that dance energy, as well as short biopics of dancers or dances that are based largely on that type of movement energy. This is in service of the goal of imparting historical knowledge through playing this game.

The two movement energies that have games currently in later stages of development are “directed forceful movement” and “light, quick movement.” The supplemental video shows the onboarding of the game, the theater (with a mannequin avatar at the moment, to be replaced soon with a Loie-like avatar) lobby and the beginnings of a greeting by Loie, and the game-in-progress that is designed to get users moving with a directed, forceful movement. What is not shown in the video is the actual avatar of Loie, the second game of “light, quick movement,” and the “live sculpture” that each user makes by the trails of their movements in virtual space.


[1] McGonigal, Jane Reality is Broken: How Games Make Us Better and How They Can Change the World. NY: Penguin Books 2011

[2] McGonigal, p. 30

Mycelial–a new collaboration

We started a new collaboration with Chicago-based choreographer and director Erica Mott. The work is entitled Mycelial: Networked Bodies, which uses the incredible life-giving and communicative capabilities of underground networks of forest mushrooms as a metaphor for the interconnectedness of various fractured communities. To that end, we are working on having mobile devices running LAIT be able to be assigned to different groups, and to have those groups respond to specific messages in real time. Watch the video of the first week’s workshop:

https://youtu.be/nTM1dmhk4ss