There’s the age-old adage that one cannot understand the experience of another unless you’ve walked a mile in their shoes. But it’s not often that we are willing or able to physically remove ourselves from personal comforts to truly empathize with others. At the New Frontier Exhibits at the 2019 Sundance Film Festival, there was considerable enthusiasm for technologies that allow people to replicate engagement with the physical world. As Chris Milk puts it in his TED talk, virtual reality (VR) is “the ultimate empathy machine.”
Experiences are what delineate us as humans, so it comes as no surprise that a powerful VR experience can be more effective than simply visualizing something. Sundance’s New Frontier allows interactive storytellers to present new narrative forms using technology. In engaging with these dynamic projects, I wanted to explore whether it was possible for thoughtful, evocative, and surprising VR to truly impact and encourage empathy among us all.
At the New Frontier, attendees were treated to a multi-room, trance-like experience that functioned as a chose-your-own-adventure type feel. Spanning two locations, the larger New Frontier Central held most of the exhibition, while a smaller space nearby in the basement of The Ray theater featured a 40-set VR cinema and the remaining projects. It was clear from the structural design of the exhibition that Sundance was very intentional in creating a social intersection element that allowed different types of people from all over to talk about the various works in the same room.
The dominating installations at New Frontier Central showcased the use of projections, artificial intelligence, pseudo-holograms, and augmented reality as it alternated between projects such as “Esperpento,” “Dirtscraper,” and “The Dial.” Customized works showcased the leaps that mixed-reality experiences have made and the ways that these technologies aim to use the principle of synesthesia to mix up human senses creating a borderline trippy experience. Adding to the mind-blowing experiences was the trend of putting physical bodes inside the frame of their experiences. No longer is VR simply removing you from the real world. “Runnin’” [pictured at top], a cosmic hip-hop dance party with artist Reggie Watts at The Ray, was one of the most thrilling, mind-bending experiences I have ever had.
The Colin Farrell-narrated “Gloomy Eyes,” is a beautifully, transportive experience that truly connected with the New Frontier goal of storytelling. The first episode, which premiered at The Ray, tells the story of transcendent love between a human girl and a zombie boy in a world where zombies are considered fugitives. The ways this story is able to simultaneously capture the hope of young love with the darkest side of humanity invokes a true sense of magic.
While each of the aforementioned experiences were well worth the hype, my goal was to uncover immersive projects that pushed the boundaries of our understanding of the human experience. It was with pleasure that I was able to experience the easy-to-use 3D volumetric capture experience “REACH.” Dubbed the “godmother of VR” by Engadget, Nonny de la Peña is once again at the forefront of the VR world, in trying to make this technology more accessible.
“Most people are calling it a VR photo booth, but that’s not exactly what it is,” de la Peña explained. “It’s actually a creation platform. It’s like iMovie for volumetric VR—super simple. And then you get to publish on like YouTube. It’s volumetric YouTube and volumetric iMovie put together. And so, what we’re doing is trying to make it super simple for folks to participate in the creation process.”
Representing a new advancement in VR technology, “REACH” allows anyone to be captured and dropped into any 3D environment of their choice, giving way for attendees to step inside of the story. The added dimensions to these immersive experiences, de la Peña believes, makes VR an increasingly important medium for telling stories.
“The reason I got started in VR was I wanted to put people on scene in stories that were otherwise kind of invisible, people whose stories were invisible,” she said. “We feel like by putting you on scene we’re able to make some of these stories become better quality, more significant.”
Because VR content can produce such powerfully emotional experiences there is a growing belief among creators in the community, including de la Peña, that with the elimination of distribution barriers (as with “REACH”) anyone will be able to harness the power of VR to not simply consume stories but to create and experience them. In 2018, there was a $12.1 billion global AR/VR revenue, a 38.4% increase. de la Peña believes that this growth signals the desire from consumers to engage the interactivity of VR experiences, much like with gaming.
Still, she was clear that not everyone believes in the power of VR experiences to shape and reshape the human condition. While some believe it is simply a fad whose uniqueness will wear with time, others believe the path to a more empathic society is already within reach in our current reality. Paul Bloom of The Atlantic argues that the best “empathy machines” are books and that language gives us a sort of reproduction of what another consciousness is like, most notably in comparison to those whose experiences and beliefs are radically different from our own. But what happens when that language is extinct or when there are too few accessible texts exploring the ways that history is repeating itself?
In “Last Whispers: An Immersive Oratorio” (a project also involving Nonny de la Peña), the cross-media artist Lena Herzog isn’t creating a new reality but instead is hoping to revive those that have been lost. “Whispers” explores languages that are endangered or extinct underscoring the ways in which we are losing our linguistic diversity and ultimately the way we understand ourselves.
“Every two weeks a language dies,” a passionate Herzog exclaimed. “The reason they are falling silent is because young generations do not speak it. The reason they are falling silent is because of genocides, climate change and people getting uprooted and having to put roots in other cultures and switch to other languages when they have to join other communities. The reason they are falling silent is because of colonization. I mean why would the Americas speak Spanish? There is a very clear answer to that—it’s because of this conquest by the Spaniards. Why is that we’re speaking English here? So, the Roman Empire alone, some speculate but it’s really hard to tell because it was a while ago and there wasn’t really research, was probably responsible for some thousand languages—the Roman Empire alone, of erasing them. Why is that languages have fallen silent? Because of globalization—cultural globalization.”
“Last Whispers” is an incredible universe that encapsulates all of these endangered and extinct languages as they swirl around you. It’s an octa-phonic design created in such a way that the frequencies hit your ears with a physical presence. This technology is incredible in that it makes the listener feel as if the languages being spoken throughout the oratorio are sharing your physical space. In this way as the sounds of the voices begin to fade, you feel that presence leave you, you feel that extinction happening. This magnificent aural experience was an intentional creation by Herzog.
“A very profound philosophical dilemma that I understood about that was that it had to do with the very nature of extinction,” she said. “The very nature of extinction is silence and we understand something when we articulate it and how we articulate it. So, it’s the very crux of the problem. So then came a really interesting challenge—how do you articulate an extinction, the form of which is silence? A very sort of direct obvious answer is to sound what has gone silent. But how do you do that? How do you really make it present? So, I started to research about how do we perceive something as present neurologically and that’s when I knew I had to have a sound team that would create an 8.1 for public presentation or a binaural for personal experience in the headphone mixes because our brain perceives that sound as present and alive.”
The visuals of “Last Whispers” are just as captivating. Herzog had originally done the visuals in her original 46-minute, 2D piece but felt it necessary to allow viewers to be inside the world that she had created. Viewers become enclosed in a sphere that encircles you as the approximate place of origin for each language is geolocated on the globe. The context provided with the visuals adds to the haunting reverberations of invocation of languages lost and incantation of those that are endangered or extinct underscoring the ways in which we are losing our linguistic diversity and ultimately the way we understand ourselves.
In what proved to be one of the most popular VR experiences at the New Frontier, Academy Award-winner Roger Ross Williams and Ayesha Nadarajah co-directed a 20-minute, 360-degree virtual reality recreation of the restaurant, Ben’s Chili Bowl in “Traveling While Black.” Paying homage to Ben and Virginia Ali’s restaurant, “Traveling While Black” immerses viewers in intimate conversation in one of the most popular safe places for African American travelers in the survival guide known as The Green Book.
The experience opens with an electrifying Reverend Sandra Butler-Truesdale sitting at the counter of Ben’s Chili Bowl. She narrates how growing up in segregated Washington she was not allowed to try on clothing in departments store for fear that she wasn’t clean and would make an item contaminated for white patrons.
“I think Sandra is a dynamic, amazing character,” Williams said in conversation with the New York Times’ Brent Staples. “So, as a documentarian as soon as I met her, I thought that she has to be sort of our narrator through the piece.”
In various moments throughout the piece, viewers are forced to reckon with the solitude of night and an almost empty restaurant. There are times where there is an orange haze and commotion outside the doors from unrest while the walls are covered in projects of the brutality that can be associated with traveling as a Black person in America. The restricted nature of the experience is unnerving, and Williams’ didn’t hesitate to enthrall viewers in the sadistic reality of state sanctioned violence. There are moments that are hard to watch as they are fraught with emotional turmoil, but that is the crux of this experience.
While some of the most exciting VR experiences still require users to be tethered to a powerful computer, creative teams are working to create accessibility whether it’s through partnerships with schools and museums or giving direct access to the user as with de la Peña’s “REACH” platform. Currently, who we feel empathy for is strongly influenced by irrelevant factors such as race, attraction, and similarity, and our empathy often takes us in the wrong direction. The experiences on hand at the New Frontier at Sundance was an impressive slate of technology, art, and psychology blended together to help address the difficulties we have understanding one another. At worst. VR experiences are changing the way we experience entertainment. At best, VR simulations offer the experience of becoming—a virtual embodiment of ourselves and others, exploring the mechanisms, possibilities and implications of such experiences. As VR technology continues to grow in both popularity and content, the line between the user and the story dissolves, leading us toward a more compassionate society.